lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-12882) Eliminate excessive lambda allocation in FacetFieldProcessorByHashDV.collectValFirstPhase
Date Thu, 01 Nov 2018 19:06:00 GMT

    [ https://issues.apache.org/jira/browse/SOLR-12882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672030#comment-16672030
] 

ASF subversion and git services commented on SOLR-12882:
--------------------------------------------------------

Commit cf445ba54998710466a7c6cb489d3162d20d127a in lucene-solr's branch refs/heads/master
from [~tpunder]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cf445ba ]

SOLR-12882: Eliminate excessive lambda allocation in json facet FacetFieldProcessorByHashDV.collectValFirstPhase


> Eliminate excessive lambda allocation in FacetFieldProcessorByHashDV.collectValFirstPhase
> -----------------------------------------------------------------------------------------
>
>                 Key: SOLR-12882
>                 URL: https://issues.apache.org/jira/browse/SOLR-12882
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: Facet Module
>    Affects Versions: 7.5
>            Reporter: Tim Underwood
>            Assignee: David Smiley
>            Priority: Major
>         Attachments: start-2018-10-31_snapshot___Users_tim_Snapshots__-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png,
start_-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> The FacetFieldProcessorByHashDV.collectValFirstPhase method looks like this:
> {noformat}
> private void collectValFirstPhase(int segDoc, long val) throws IOException {
>  int slot = table.add(val); // this can trigger a rehash
>  // Our countAcc is virtual, so this is not needed:
>  // countAcc.incrementCount(slot, 1);
> super.collectFirstPhase(segDoc, slot, slotNum ->
> { Comparable value = calc.bitsToValue(val); return new SlotContext(sf.getType().getFieldQuery(null,
sf, calc.formatValue(value))); }
> );
> }
> {noformat}
>  
> For each value that is being iterated over there is a lambda allocation that is passed
as the slotContext argument to the super.collectFirstPhase method. The lambda can be factored
out such that there is only a single instance per FacetFieldProcessorByHashDV instance. 
> The only tradeoff being that the value needs to be looked up from the table in the lambda. 
However looking the value up in the table is going to be less expensive than a memory allocation
and also the slotContext lambda is only used in RelatednessAgg and not for any of the field
faceting or metrics.
>  
> {noformat}
> private void collectValFirstPhase(int segDoc, long val) throws IOException {
>   int slot = table.add(val); // this can trigger a rehash
>   // Our countAcc is virtual, so this is not needed:
>   // countAcc.incrementCount(slot, 1);
>   super.collectFirstPhase(segDoc, slot, slotContext);
> }
> /**
>  * SlotContext to use during all {@link SlotAcc} collection.
>  *
>  * This avoids a memory allocation for each invocation of collectValFirstPhase.
>  */
> private IntFunction<SlotContext> slotContext = (slotNum) -> {
>   long val = table.vals[slotNum];
>   Comparable value = calc.bitsToValue(val);
>   return new SlotContext(sf.getType().getFieldQuery(null, sf, calc.formatValue(value)));
> };
> {noformat}
>  
> FacetFieldProcessorByArray already follows this same pattern



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message