lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Audenaerde <rob.audenae...@gmail.com>
Subject Re: GROUP BY in Lucene
Date Mon, 10 Aug 2015 12:56:55 GMT
You can write a custom (facet) collector to do this. I have done something
similar, I'll describe my approach:

For all the values that need grouping or aggregating, I have added a
FacetField ( an AssociatedFacetField, so I can store the value alongside
the ordinal) . The main search stays the same, in your case for example a
NumericRangeQuery  (if the date is store in ms).

Then I have a custom facet collector that does the grouping.

Basically, it goes through all the MatchingDocs. For each doc, it creates a
unique key (composed of X, Y and Z), and makes aggregates as needed (sum
D).These are stored in a map. If a key is already in the map, the existing
aggregate is added to the new value. Tricky is to make your unique key fast
and immutable, so you can  precompute the hashcode.

This is fast enough if the number of unique keys is smallish (<10.000),
index size +- 1M docs).

-Rob


On Mon, Aug 10, 2015 at 2:47 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> Lucene has a grouping module that has several approaches for grouping
> search hits, though it's only by a single field I believe.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Sun, Aug 9, 2015 at 2:55 PM, Gimantha Bandara <gimantha@wso2.com>
> wrote:
> > Hi all,
> >
> > Is there a way to achieve $subject? For example, consider the following
> SQL
> > query.
> >
> > SELECT A, B, C SUM(D) as E FROM  `table` WHERE time BETWEEN fromDate AND
> > toDate *GROUP BY X,Y,Z*
> >
> > In the above query we can group the records by, X,Y,Z. Is there a way to
> > achieve the same in Lucene? (I guess Faceting would help, But is it
> > possible get all the categoryPaths along with the matching records? ) Is
> > there any other way other than using Facets?
> >
> > --
> > Gimantha Bandara
> > Software Engineer
> > WSO2. Inc : http://wso2.com
> > Mobile : +94714961919
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message