lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Harwood (JIRA)" <>
Subject [jira] Commented: (LUCENE-584) Decouple Filter from BitSet
Date Tue, 04 Dec 2007 10:50:43 GMT


Mark Harwood commented on LUCENE-584:

For the data structures (bitset/openbitset/sorted VintList/) I would suggest one of these:
IntSet, IntegerSet or IntegerSequence as names for the common interface.
I did a quick Google for IntegerSet and you are in the number one spot, Paul :) []

// A cachable, immutable, sorted, threadsafe collection of ints.
interface IntegerSet
   IntegerSetIterator getIterator();
   int size(); //negative numbers could be used to represent estimates?

// A single-use thread-unsafe iterator.
interface IntegerSetIterator
    boolean next();
    boolean skipTo(int next);
    int currentValue();

If _detailed_ explanations of hits are required these should really sit with the source not
the result- i.e. with the Filters. They contain all the match criteria used to populate IntegerSets
 and can be thought of more generically as IntegerSetBuilder. 

//Contains criteria to create a set of matching documents. MUST implement hashcode and equals
based on this criteria to enable use as cache keys for IntegerSets.
interface IntegerSetBuilder extends Serializable
      IntegerSet build (IndexReader reader)
      Explanation explain(int docNr);

A single CachingIntegerSetBuilder class would be able to take ANY IntegerSetBuilder as a source,
cache ANY type of IntegerSet they produced and defer back to the original IntegerSetBuilder
for a full and thorough explanation of a match even when the match occurred on a cached IntegerSet,
if required.

class CachingIntegerSetBuilder implements IntegerSetBuilder
     private WeakHashMap perIndexReaderCache;
     public CachingIntegerSetBuilder(IntegerSetBuilder delegate) {....}

The reason for introducing IntegerSetBuilder as a more generic name than "Filter" is IntegerSets
have uses outside of  filtering e.g. to do category counts or clustering. In these use cases
they don't actually perform anything to do with filtering.  It may actually be better named
DocIdSetBuilder given that it is tied to Lucene's IndexReader and therefore limited to producing
sets of document ids.

> Decouple Filter from BitSet
> ---------------------------
>                 Key: LUCENE-584
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.0.1
>            Reporter: Peter Schäfer
>            Assignee: Michael Busch
>            Priority: Minor
>         Attachments: bench-diff.txt, bench-diff.txt, lucene-584-take2.patch, lucene-584.patch,
Matcher-20070905-2default.patch, Matcher-20070905-3core.patch, Matcher-20071122-1ground.patch,
> {code}
> package;
> public abstract class Filter implements 
> {
>   public abstract AbstractBitSet bits(IndexReader reader) throws IOException;
> }
> public interface AbstractBitSet 
> {
>   public boolean get(int index);
> }
> {code}
> It would be useful if the method =Filter.bits()= returned an abstract interface, instead
of =java.util.BitSet=.
> Use case: there is a very large index, and, depending on the user's privileges, only
a small portion of the index is actually visible.
> Sparsely populated =java.util.BitSet=s are not efficient and waste lots of memory. It
would be desirable to have an alternative BitSet implementation with smaller memory footprint.
> Though it _is_ possibly to derive classes from =java.util.BitSet=, it was obviously not
designed for that purpose.
> That's why I propose to use an interface instead. The default implementation could still
delegate to =java.util.BitSet=.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message