lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <>
Subject [jira] Commented: (LUCENE-2837) Collapse Searcher/Searchable/IndexSearcher; remove contrib/remote; merge PMS into IndexSearcher
Date Tue, 28 Dec 2010 22:53:45 GMT


Uwe Schindler commented on LUCENE-2837:

I like that! +1

We should discuss about how many threads should be spawned. If you have an index with many
segments, even small ones, I think only the larger segments should be separate threads, all
others should be handled sequentially. So maybe add a maxThreads cound, then sort the IndexReaders
by maxDoc and then only spawn maxThreads-1 threads for the bigger readers and then one additional
thread for the rest?

> Collapse Searcher/Searchable/IndexSearcher; remove contrib/remote; merge PMS into IndexSearcher
> -----------------------------------------------------------------------------------------------
>                 Key: LUCENE-2837
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Michael McCandless
>             Fix For: 4.0
>         Attachments: LUCENE-2837.patch
> We've discussed cleaning up our *Searcher stack for some time... I
> think we should try to do this before releasing 4.0.
> So I'm attaching an initial patch which:
>   * Removes Searcher, Searchable, absorbing all their methods into IndexSearcher
>   * Removes contrib/remote
>   * Removes MultiSearcher
>   * Absorbs ParallelMultiSearcher into IndexSearcher (ie you can now
>     pass useThreads=true, or a custom ES to the ctor)
> The patch is rough -- I just ripped stuff out, did search/replace to
> IndexSearcher, etc.  EG nothing is directly testing using threads with
> IndexSearcher, but before committing I think we should add a
> newSearcher to LuceneTestCase, which randomly chooses whether the
> searcher uses threads, and cutover tests to use this instead of making
> their own IndexSearcher.
> I think MultiSearcher has a useful purpose, but as it is today it's
> too low-level, eg it shouldn't be involved in rewriting queries: the
> Query.combine method is scary.  Maybe in its place we make a higher
> level class, with limited API, that's able to federate search across
> multiple IndexSearchers?  It'd also be able to optionally use thread
> per IndexSearcher.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message