lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <>
Subject Re: [jira] Commented: (LUCENE-565) Supporting deleteDocuments in IndexWriter (Code and Performance Results Provided)
Date Wed, 06 Sep 2006 19:40:41 GMT

On Sep 6, 2006, at 12:06 PM, Yonik Seeley wrote:

> Hmmm, not rewriting stored fields is nice.
> I guess that could apply to anything that's strictly document
> specific, such as term vectors.

Yes.  Remember the old benchmarks I posted a few months ago?   
KinoSearch's performance was much closer to Lucene when fields and  
term vectors where turned on.  This is why.

> One problem with extending it to any number of documents is that the
> complexity goes up because you can't assume it will all fit in memory.

That problem is 100% solved by the external sorter.  In fact, that's  
the whole reason that the external sorter is needed.

> Still, how do you deal with multiple sessions w/o being able to  
> merge segments?
> Do you just keep creating more and more segments?  It seems like if
> you had a way to read a segment into an existing "big bucket", then
> that's a segment merger.

The relevant classes have an add_segment() method.  Aspects are  
similar to SegmentMerger.

Marvin Humphrey
Rectangular Research

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message