lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Bernstein (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-8909) Streaming Expressions should leverage streaming facets
Date Sat, 26 Mar 2016 02:15:25 GMT

    [ https://issues.apache.org/jira/browse/SOLR-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15212736#comment-15212736
] 

Joel Bernstein edited comment on SOLR-8909 at 3/26/16 2:15 AM:
---------------------------------------------------------------

This is another powerful tool in the toolbox.

We can probably build this into the FacetStream by adding the method param and a new code
path to handle the merge.

With the SQL handler, we can probably use this approach in most scenarios because we can re-order
the Tuples by wrapping the FacetStream in a RankStream.

The RollupStream will likely only need to be used following distributed joins.



was (Author: joel.bernstein):
This is another powerful tool in toolbox.

We can probably build this into the FacetStream by adding the method param and a new code
path to handle the merge.

With the SQL handler, we can probably use this approach in most scenarios because we can re-order
the Tuples by wrapping the FacetStream in a RankStream.

The RollupStream will likely be only need to be used following distributed joins.


> Streaming Expressions should leverage streaming facets
> ------------------------------------------------------
>
>                 Key: SOLR-8909
>                 URL: https://issues.apache.org/jira/browse/SOLR-8909
>             Project: Solr
>          Issue Type: Bug
>            Reporter: Yonik Seeley
>
> The JSON Facet API can currently stream facets (use method=stream) from a single node.
 Each facet bucket is calculated as it is written out, so field cardinality has no effect
on memory.
> This is only from a single node - normal distributed search/faceting does not stream...
 But that's what streaming expressions are for anyway!
> One current caveat: streaming currently only works with "sort=index asc" (the term order
in the Lucene index).
> Future work could allow more complex sorts, at the cost of some memory to calculate the
sort criteria for each bucket prior to streaming out.  Of course more complex sorts would
require more complex merging logic (i.e. even a sort by bucket count is not a simple merge
sort and requires more buffering in the merging node).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message