lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <>
Subject Re: [jira] Commented: (LUCENE-140) docs out of order
Date Tue, 16 Jan 2007 22:23:02 GMT

On Jan 12, 2007, at 3:57 AM, Michael McCandless wrote:

> Chris Hostetter wrote:
>> : I think we should deprecate the "create" argument to
>> : FSDirectory.getDirectory(*) and leave only the create argument in
>> : IndexWriter's constructors.  Am I missing something?  Is there  
>> are a
>> : reason not to do this?
>> i actual wonder about hte problem from the oposite direction: to  
>> me it
>> makes sense that FSDirectory has a "create" option, and it makes  
>> sense
>> that "new IndexWriter(File path, Analyzer a, boolean create)"  
>> exists since
>> it's suppose to be a convinience method for
>> "new IndexWriter(FSDirectory.getDirectory(path,create), Analyzer  
>> a)" ...
>> but what does "new IndexWriter(Directory d, Analyzer a, boolean  
>> create)"
>> exist? ... if people are dealing with a Directory object directly,  
>> then
>> they can specify create when they access the Directory object right?
>> i note that there are versions of FSDirectory.getDirectory which  
>> take in a
>> boolean named "doRemoveOldFiles" and the versions of IndexWriter
>> that call FSDirectory.getDirectory allways pass "false" to that
>> value ... perhaps that should change?
> Good points Hoss!
> We could indeed choose instead to deprecate IndexWriter's "create"
> (except for the convenience methods, I agree) and move all creation
> logic down to the Directory layer.

You're both right.  :)  IMO, index creation logic doesn't belong in  
either IndexWriter or Directory. :)

> The challenge is, "creating" an index is not really as simple an
> operation as just "removing all old files":
>   * You need to at least retry on failure since readers may be using
>     the index (FSDirectory doesn't do this now, but, IndexFileDeleter
>     does) on Windows.
>   * You must also take care not to harm / conflict with any readers
>     that are currently using the old index.  With lockless, we can now
>     "create" an index into an index directory that readers are using
>     on Windows (this is why I added the doRemoveOldFiles param to the
>     FSDirectory.getDirectory(*) methods).  This allows a use case of
>     re-creating your index from scratch while readers are using the
>     old one.
>   * Looking forward to issues like LUCENE-710, even more knowledge
>     will be required to know what not to delete (and IndexFileDeleter
>     would have been extended with this knowledge).

How many people expect to be able to continue from an index they just  
clobbered?  That seems counterintuitive, and I don't think any  
complexity should be added to support it.

Marvin Humphrey
Rectangular Research

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message