lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Laban (JIRA)" <>
Subject [jira] [Created] (SOLR-8559) FCS facet performance optimization
Date Sat, 16 Jan 2016 23:50:39 GMT
Keith Laban created SOLR-8559:

             Summary: FCS facet performance optimization
                 Key: SOLR-8559
             Project: Solr
          Issue Type: Bug
            Reporter: Keith Laban

While profiling a large collection (multi-sharded billions of documents), I found that a fast
(5-10ms query) would take 20-30 seconds when doing facets even when {{facet.mincount=1}}

Profiling made it apparent that with {{facet.method=fcs}} 99% of the time was [spent here|].

{{queue.udpateTop}} gets called {{numOfSegments*numTerms}}, the worst case when every term
is in every segment. This formula doesn't take into account whether or not any of the terms
have a positive count with respect to the docset.

These optimizations are aimed to do two things:

# When mincount>0 don't include segments which all terms have zero counts. This should
significantly speed up processing when terms are high cardinality and the matching docset
is small

# FIXED TODO optimization: when mincount>0 move segment position the next non zero term

both of these changes will minimize the number of called needed to the slow {{updateTop}}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message