lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Terry Smith <sheb...@gmail.com>
Subject Re: How to read multiple indices in parallel.
Date Tue, 07 Apr 2015 14:47:04 GMT
Gimantha,

Search will run in parallel even across indices.

This happens because IndexSearcher searches by LeafReader and it doesn't
matter where those LeafReaders come from (DirectoryReader or MultiReader)
they are all treated equally.

Example:

DirectoryReader(A):
    LeafReader(B), LeafReader(C)

DirectoryReader(D):
    LeafReader(E), LeafReader(F)

Searching over A would use leaves B, C.
Searching over B would use leaves E, F.
Searching over a MultiReader on (A, B) would use leaves B, C, E, F.

This runs serially by default but can run in parallel if you provide an
ExecutorService to the IndexSearcher and use a compatible search() method
on it.

--Terry


On Tue, Apr 7, 2015 at 10:27 AM, Gimantha Bandara <gimantha@wso2.com> wrote:

> Hi Terry,
>
> I have multiple indices in separate locations. If I used multireader and
> used an executorservice with the indexSearcher It will go thru the segments
> in parallel and search right? But still searching between different indices
> will happen sequentially..Isnt it?
>
> On Tue, Apr 7, 2015 at 7:15 PM, Terry Smith <shebiki@gmail.com> wrote:
>
> > Gimantha,
> >
> > With Lucene 5.0 you can pass in an ExecutorService to the constructor of
> > your IndexSearcher and it will search the segments in parallel if you use
> > one of the IndexSearcher.search() methods that returns a TopDocs (and
> don't
> > supply your own Collector).
> >
> > The not-yet-released Lucene 5.1 includes some changes (LUCENE-6294
> > <https://issues.apache.org/jira/browse/LUCENE-6294>) that enable better
> > parallel query support.
> >
> > --Terry
> >
> >
> > On Tue, Apr 7, 2015 at 8:09 AM, Gimantha Bandara <gimantha@wso2.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > As I can see the Multireader is reading the multiple indices
> sequentially
> > > (correct me if I am wrong). So using a IndexSearcher on a multireader
> > will
> > > also perform sequential searches right? Is there a lucene-built-in
> class
> > to
> > > search several indices parallely?
> > >
> > > --
> > > Gimantha Bandara
> > > Software Engineer
> > > WSO2. Inc : http://wso2.com
> > > Mobile : +94714961919
> > >
> >
>
>
>
> --
> Gimantha Bandara
> Software Engineer
> WSO2. Inc : http://wso2.com
> Mobile : +94714961919
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message