lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Taylor <>
Subject Re: Lucene Merge failing on Open Files
Date Mon, 04 Apr 2011 19:59:46 GMT
On 04/04/2011 20:13, Michael McCandless wrote:
> How are you merging these indices?  (IW.addIndexes?).
> Are you changing any of IW's defaults, eg mergeFactor?
> Mike
Hi Mike

I have


these are a hangover from earlier code, I tried changing them and it 
didnt seem to make any difference, but do they look wrong ?

The index that falls over during optmizationcreates about 10,000,000 records

What I do is build 10 different indexes sequentially one after the after 
(so only one is hitting the db at any one time) but once the index is 
built I optimize the index in the background
by creating an instance of the following class and submitting it to  an 
ExecutionService , configured to have at most 2 threads acting

I built my own class rather than just using optimize() with the 
background option because that wouldnt allow me to do the necessary 

static class IndexWriterOptimizerAndClose implements Callable<Boolean>
         private int             maxId;
         private IndexWriter     indexWriter;
         private DatabaseIndex   index;
         private IndexOptions    options;

          * @param maxId
          * @param indexWriter
          * @param index
          * @param options
         public IndexWriterOptimizerAndClose(int maxId, IndexWriter 
indexWriter, DatabaseIndex index, IndexOptions options)
             this.indexWriter= indexWriter;

         public Boolean call() throws IOException, SQLException

             StopWatch clock = new StopWatch();
             String path = options.getIndexesDir() + index.getFilename();
             System.out.println(index.getName()+":Started Optimization 
at "+Utils.formatCurrentTimeForOutput());

             // For debugging to check sql is not creating too few/many rows
             if(true) {
                 int dbRows = index.getNoOfRows(maxId);
                 IndexReader reader = File(path)),true);
                 System.out.println(index.getName()+":"+dbRows+" db 
rows:"+(reader.maxDoc() - 1)+" lucene docs");
Optimization:" + Utils.formatClock(clock));

             return true;

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message