lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <>
Subject [jira] [Updated] (SOLR-6349) LocalParams for enabling/disabling individual stats
Date Fri, 27 Feb 2015 00:11:05 GMT


Hoss Man updated SOLR-6349:
    Attachment: SOLR-6349.patch

Two steps forward, one step back: I fixed the bugs i mentioned before dealing with localparam
stats that are all 'false', and cleaned up a bit of the stats returned/calculated logic --
this lead me to understanding that the "whitebox" isShard test i was confused by before was
definitely broken (see below) ... i tried to use SOLR-7147 to do a true whitebox test and
inspect _real_ shard requests/responses, but got blocked by some bugs in the test framework

Full notes about changes in this patch...

* StatsField
** add Set<Stat> DEFAULT_STATS
** populateStatsRequested(...)
*** rename to populateStatsSets() and eliminate args -- method is already mucking with other
state directly
*** use DEFAULT_STATS if and only if there are no stat localparams (even if false)
** calculateStats()
*** fix to not assume anything about the empty set - empty set means calculate nothing

* StatsComponentTest
** testIndividualStatLocalParams
*** my review & bug fixing in StatsField confirmed that the whitebox testing here wasn
bogus - the types of "isShard=true" requests simulated here don't actaully look like what
a real shard request looks like.  This test _was_ asking for each of the individual stats
that appear as in getDistribDeps() of any stat in an isShard request and confirmed that only
that single stat was returned -- but the actual code doesn't ask for individual distribDeps
that way.  it asks for the oroginally requested stat and trusts that the isShard indication
will cause only the deps to be returned
*** fixed the whitebox queries to simulate the real requests for every stats, and assert all
deps come back - even if the original request excluded those deps 

* TestDistributedSearch
** beefed up testing of distrib requests where the deps of a stat requested by the client
are also explicitly excluded by the client (ie: "mean=true sum=false count=false"
** started down the path of leveraging SOLR-7147 to inspect the shard req/rsp to ensure we
only get what we need - but got blocked by SOLR-7171 (see nocommits)

> LocalParams for enabling/disabling individual stats
> ---------------------------------------------------
>                 Key: SOLR-6349
>                 URL:
>             Project: Solr
>          Issue Type: Sub-task
>            Reporter: Hoss Man
>         Attachments: SOLR-6349-tflobbe.patch, SOLR-6349-tflobbe.patch, SOLR-6349-tflobbe.patch,
SOLR-6349-xu.patch, SOLR-6349-xu.patch, SOLR-6349-xu.patch, SOLR-6349-xu.patch, SOLR-6349.patch,
SOLR-6349.patch, SOLR-6349.patch, SOLR-6349.patch, SOLR-6349.patch, SOLR-6349___bad_idea_broken.patch,
> Stats component currently computes all stats (except for one) every time because they
are relatively cheap, and in some cases dependent on eachother for distrib computation --
but if we start layering stats on other things it becomes unnecessarily expensive to compute
all the stats when they just want the "sum" (and it will definitely become excessively verbose
in the responses).  
> The plan here is to use local params to make this configurable.  All of the existing
stat options could be modeled as a simple boolean param, but future params (like percentiles)
might take in a more complex param value...
> Example:
> {noformat}
> stats.field={!min=true max=true percentiles='99,99.999'}price
> stats.field={!mean=true}weight
> {noformat}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message