lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: [jira] Commented: (LUCENE-140) docs out of order
Date Fri, 12 Jan 2007 11:57:09 GMT
Chris Hostetter wrote:
> : I think we should deprecate the "create" argument to
> : FSDirectory.getDirectory(*) and leave only the create argument in
> : IndexWriter's constructors.  Am I missing something?  Is there are a
> : reason not to do this?
> i actual wonder about hte problem from the oposite direction: to me it
> makes sense that FSDirectory has a "create" option, and it makes sense
> that "new IndexWriter(File path, Analyzer a, boolean create)" exists since
> it's suppose to be a convinience method for
> "new IndexWriter(FSDirectory.getDirectory(path,create), Analyzer a)" ...
> but what does "new IndexWriter(Directory d, Analyzer a, boolean create)"
> exist? ... if people are dealing with a Directory object directly, then
> they can specify create when they access the Directory object right?
> i note that there are versions of FSDirectory.getDirectory which take in a
> boolean named "doRemoveOldFiles" and the versions of IndexWriter
> that call FSDirectory.getDirectory allways pass "false" to that
> value ... perhaps that should change?

Good points Hoss!

We could indeed choose instead to deprecate IndexWriter's "create"
(except for the convenience methods, I agree) and move all creation
logic down to the Directory layer.

The challenge is, "creating" an index is not really as simple an
operation as just "removing all old files":

   * You need to at least retry on failure since readers may be using
     the index (FSDirectory doesn't do this now, but, IndexFileDeleter
     does) on Windows.

   * You must also take care not to harm / conflict with any readers
     that are currently using the old index.  With lockless, we can now
     "create" an index into an index directory that readers are using
     on Windows (this is why I added the doRemoveOldFiles param to the
     FSDirectory.getDirectory(*) methods).  This allows a use case of
     re-creating your index from scratch while readers are using the
     old one.

   * Looking forward to issues like LUCENE-710, even more knowledge
     will be required to know what not to delete (and IndexFileDeleter
     would have been extended with this knowledge).

So basically because "creation" has become / is becoming an operation
that needs at least some decent understanding of the index file
format, it feels to me like it belongs at the IndexWriter layer and
not the Directory layer.  If we move "creation" entirely into
IndexWriter, then the Directory would then become a clean layer that
exposes the interface for making changes but does not itself try to do
any changes.

Also, we have other interesting Directory implementations
(MMapDirectory, DbDirectory, etc.); if we leave "create" at the
Directory layer then each of these places will have to replicate the
logic of "what it takes to cleanly create an index".


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message