lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <>
Subject Java 1.5 was [jira] Updated: (LUCENE-600) ParallelWriter companion to ParallelReader
Date Tue, 13 Jun 2006 13:35:53 GMT
Well, we have our first Java 1.5 patch...  Now that we have had a week 
or two to digest the comments, do we want to reopen the discussion?

Chuck Williams (JIRA) wrote:
>      [ ]
> Chuck Williams updated LUCENE-600:
> ----------------------------------
>     Attachment: ParallelWriter.patch
> Patch to create and integrate ParallelWriter, Writable and TestParallelWriter -- also
modifies build to use java 1.5.
>> ParallelWriter companion to ParallelReader
>> ------------------------------------------
>>          Key: LUCENE-600
>>          URL:
>>      Project: Lucene - Java
>>         Type: Improvement
>>   Components: Index
>>     Versions: 2.1
>>     Reporter: Chuck Williams
>>  Attachments: ParallelWriter.patch
>> A new class ParallelWriter is provided that serves as a companion to ParallelReader.
 ParallelWriter meets all of the doc-id synchronization requirements of ParallelReader, subject
>>     1.  ParallelWriter.addDocument() is synchronized, which might have an adverse
effect on performance.  The writes to the sub-indexes are, however, done in parallel.
>>     2.  The application must ensure that the ParallelReader is never reopened inside
ParallelWriter.addDocument(), else it might find the sub-indexes out of sync.
>>     3.  The application must deal with recovery from ParallelWriter.addDocument()
exceptions.  Recovery must restore the synchronization of doc-ids, e.g. by deleting any trailing
document(s) in one sub-index that were not successfully added to all sub-indexes, and then
optimizing all sub-indexes.
>> A new interface, Writable, is provided to abstract IndexWriter and ParallelWriter.
 This is in the same spirit as the existing Searchable and Fieldable classes.
>> This implementation uses java 1.5.  The patch applies against today's svn head. 
All tests pass, including the new TestParallelWriter.


Grant Ingersoll 
Sr. Software Engineer 
Center for Natural Language Processing 
Syracuse University 
School of Information Studies 
335 Hinds Hall 
Syracuse, NY 13244 
Voice:  315-443-5484 
Fax: 315-443-6886 

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message