lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From robert engels <>
Subject detected corrupted index / performance improvement
Date Wed, 06 Feb 2008 06:45:28 GMT
I had a recent sidebar with another user, and it got me to thinking.

Do we have any way of determining if a segment is definitely OK/VALID ?

If so, a much more efficient transactional system could be developed.

Serialize the updates to a log file. Sync the log. Update the lucene  
index WITHOUT any sync.  Log file writing/sync is VERY efficient  
since it is sequential, and a single file.

Upon open of the index, detect if index was not shutdown cleanly. If  
so, determine the last valid segment, delete the bad segments, and  
then perform the updates (from the log file) since the last valid  
segment was written.

The detection could be a VERY slow operation, but this is ok, since  
it should be rare, and then you will only pay this price on the rare  
occasion, not on every update.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message