lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Laban (JIRA)" <>
Subject [jira] [Commented] (SOLR-8559) FCS facet performance optimization
Date Fri, 22 Jan 2016 21:20:39 GMT


Keith Laban commented on SOLR-8559:

After a doing some performance testing, this change shows about 4 orders of magnitudes speed

For a collection with 100M documents a query matching 2,675 documents and 1,964 unique terms
(facet.mincount=1&facet.limit=-1&facet.method=fcs) sped up query performance from
11664ms (11 seconds) to 30ms. 

> FCS facet performance optimization
> ----------------------------------
>                 Key: SOLR-8559
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>          Components: faceting
>    Affects Versions: 5.4, Trunk
>            Reporter: Keith Laban
>            Assignee: Dennis Gove
>              Labels: optimization, performance
>             Fix For: Trunk
>         Attachments: SOLR-8559-4-10-4.patch, SOLR-8559.patch, solr-8559.patch
> While profiling a large collection (multi-sharded billions of documents), I found that
a fast (5-10ms query) which had no matches would take 20-30 seconds when doing facets even
when {{facet.mincount=1}}
> Profiling made it apparent that with {{facet.method=fcs}} 99% of the time was [spent
> {{queue.udpateTop}} gets called {{numOfSegments*numTerms}}, the worst case when every
term is in every segment. This formula doesn't take into account whether or not any of the
terms have a positive count with respect to the docset.
> These optimizations are aimed to do two things:
> # When mincount>0 don't include segments which all terms have zero counts. This should
significantly speed up processing when terms are high cardinality and the matching docset
is small
> # FIXED TODO optimization: when mincount>0 move segment position the next non zero
term value.
> both of these changes will minimize the number of called needed to the slow {{updateTop}}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message