lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peng Gao <p...@esri.com>
Subject RE: Accumulating facets over a MultiReader
Date Wed, 03 Jul 2013 15:14:52 GMT
Hi Shai,
Thanks for the reply.
Yes I used a single TaxonomyReader instance. 
I am adding facets to an existing app, which maintains two indexes, one for indexing
system tools, and the other indexing user data in folders.
The system tool index contains docs for describing the tool usage, and etc, which needs to
be its own 
index.

It turned out that my problem is not MultiReader. The problem is the index, i.e. the way it's
created.
The app crawls folders in multiple threads, and each thread creates a temp index.
The main thread merges the temp indexes into the master index, using IndexWriter.AddIndexes().
If the temp index has facet index, this approach creates a bad index.

Is there a way I can build faceted index in multiple threads?

- Gao Peng

> -----Original Message-----
> From: Shai Erera [mailto:serera@gmail.com]
> Sent: Monday, July 01, 2013 8:25 PM
> To: java-user@lucene.apache.org
> Subject: Re: Accumulating facets over a MultiReader
> 
> Hi,
> 
> I assume that you use a single TaxonomyReader instance? It must be the same
> for both indexes, that is, both indexes must share the same taxonomy index,
> or otherwise their ordinals would not match as well as you may hit such
> exceptions since one index may have bigger ordinals than what the taxonomy
> reader knows about.
> 
> Can you share a little bit about your scenario and why do you need to use a
> MultiReader?
> 
> Shai
> 
> 
> 
> On Tue, Jul 2, 2013 at 3:31 AM, Peng Gao <pgao@esri.com> wrote:
> 
> > How do I accumulate counts over a MultiReader (2 IndexReader)?
> > The following code causes an IOException:
> >
> >       ArrayList<FacetRequest> facetRequests = new
> > ArrayList<FacetRequest>();
> >       for (String groupField : groupFields)
> >         facetRequests.add(new CountFacetRequest(new
> > CategoryPath(groupField, '/'), 1));
> >
> >       FacetSearchParams facetSearchParams = new
> > FacetSearchParams(facetRequests);
> >       StandardFacetsAccumulator accumulator = new
> > StandardFacetsAccumulator(facetSearchParams, reader, taxonomyReader);
> >       FacetsCollector facetsCollector =
> > FacetsCollector.create(accumulator);
> >
> >       // perform documents search and facets accumulation
> >       searcher.search(query, facetsCollector);
> >
> >       // return facets results in a proper format
> >       return getFacetResults(facetsCollector, sr);
> >
> >
> > Here reader is a MultiReader of 2. I am using Lucene 4.3.1.
> >
> > The following is the callstack. It looks like it has something to do
> > with the MultiReader.
> > How do I make it work?
> >
> >
> > java.io.IOException: PANIC: Got unexpected exception while trying to
> > get/calculate total counts
> >       at
> >
> org.apache.lucene.facet.search.StandardFacetsAccumulator.accumulate(Standar
> dFacetsAccumulator.java:156)
> >       at
> >
> org.apache.lucene.facet.search.StandardFacetsAccumulator.accumulate(Standar
> dFacetsAccumulator.java:378)
> >       at
> >
> org.apache.lucene.facet.search.FacetsCollector.getFacetResults(FacetsCollec
> tor.java:214)
> >       at
> >
> com.esri.arcgis.search.SearchHandler.getFacetResults(SearchHandler.java:551
> )
> >       at
> > com.esri.arcgis.search.SearchHandler.search(SearchHandler.java:350)
> >       at
> > com.esri.arcgis.search.SearchHandler.search(SearchHandler.java:239)
> >       at
> > com.esri.arcgis.search.test.Searcher.invokeSearch(Searcher.java:58)
> >       at com.esri.arcgis.search.test.Searcher.main(Searcher.java:32)
> > Caused by: java.lang.ArrayIndexOutOfBoundsException: 34
> >       at
> >
> org.apache.lucene.facet.search.CountingAggregator.aggregate(CountingAggrega
> tor.java:43)
> >       at
> >
> org.apache.lucene.facet.search.StandardFacetsAccumulator.fillArraysForParti
> tion(StandardFacetsAccumulator.java:309)
> >       at
> >
> org.apache.lucene.facet.search.StandardFacetsAccumulator.accumulate(Standar
> dFacetsAccumulator.java:168)
> >       at
> >
> org.apache.lucene.facet.complements.TotalFacetCounts.compute(TotalFacetCoun
> ts.java:176)
> >       at
> >
> org.apache.lucene.facet.complements.TotalFacetCountsCache.computeAndCache(T
> otalFacetCountsCache.java:157)
> >       at
> >
> org.apache.lucene.facet.complements.TotalFacetCountsCache.getTotalCounts(To
> talFacetCountsCache.java:104)
> >       at
> >
> org.apache.lucene.facet.search.StandardFacetsAccumulator.accumulate(Standar
> dFacetsAccumulator.java:129)
> >       ... 7 more
> >
> >
> >
> >
> >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message