lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Weiss <dawid.we...@cs.put.poznan.pl>
Subject Re: (SOLR-5488) Fix up test failures for Analytics Component
Date Thu, 21 Nov 2013 22:15:18 GMT
... and sure the problem does reproduce 100% if you use IBM J9, as in
the failed build. I would fix the bug first (don't know how :), using
J9 to diagnose the problem, then correct the code not to use the
HashMap order.

Dawid

On Thu, Nov 21, 2013 at 11:13 PM, Dawid Weiss
<dawid.weiss@cs.put.poznan.pl> wrote:
> Jira is down, so here:
>
> Just looking randomly at the code (pretty hairy...) I noticed this in
> TestHarness.java
>
> {code}
>   public class LocalRequestFactory {
>     public String qtype = null;
>     public int start = 0;
>     public int limit = 1000;
>     public Map<String,String> args = new HashMap<String,String>();
> {code}
>
> We should *not* be using HashMaps for anything that later on requires
> iteration over map elements (as this is the case here) because hash
> maps are jvm-dependent and pretty much don't have a reliable iteration
> order. This should be changed to LinkedHashMap.
>
> D.
>
> On Thu, Nov 21, 2013 at 11:04 PM, Dawid Weiss (JIRA) <jira@apache.org> wrote:
>>
>>     [ https://issues.apache.org/jira/browse/SOLR-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13829396#comment-13829396
]
>>
>> Dawid Weiss commented on SOLR-5488:
>> -----------------------------------
>>
>> There is something seriously odd with this test -- I'm guessing it depends on previous
context somehow.
>>
>> When I re-run it with the same seed I get consistently a different set of numbers.
The above are clearly wrong because these raw bits correspond to:
>>
>> {code}
>> 263901.0 - 27.72225134241226 < ...
>> {code}
>>
>> When I look at the dumped log the most recent query is also consistent in my runs:
>>
>> {code}
>> oasc.SolrCore.execute [collection1] webapp=null path=null params={o.cr.s.long_ld=count(long_ld)&o.ur.s.double_dd=unique(double_dd)&o.str.s.int_id=stddev(int_id)&o.p6r.s.string_sd=percentile(60,string_sd)&o.p2r.s.string_sd=percentile(20,string_sd)&o.mr.s.double_dd=mean(double_dd)&o.st.s.float_fd=stddev(float_fd)&o.mar.s.float_fd=max(float_fd)&o.sosr.s.long_ld=sumofsquares(long_ld)&o.ur.s.date_dtd=unique(date_dtd)&o.ur.s.string_sd=unique(string_sd)&o.mir.s.float_fd=min(float_fd)&o.misr.s.int_id=missing(int_id)&o.misr.s.double_dd=missing(double_dd)&o.p2r.s.int_id=percentile(20,int_id)&o.medr.s.date_dtd=median(date_dtd)&o.p6r.s.int_id=percentile(60,int_id)&indent=true&o.ur.s.long_ld=unique(long_ld)&o.misr.s.string_sd=missing(string_sd)&o.sosr.s.float_fd=sumofsquares(float_fd)&o.misr.s.date_dtd=missing(date_dtd)&o.mr.s.long_ld=mean(long_ld)&o.medr.s.long_ld=median(long_ld)&o.str.s.double_dd=stddev(double_dd)&o.p2r.s.float_fd=percentile(20,float_fd)&o.cr.s.float_fd=count(float_fd)&o.p6r.s.float_fd=percentile(60,float_fd)&o.sr.s.double_dd=sum(double_d)&o.mr.s.float_fd=mean(float_fd)&o.mir.s.long_ld=min(long_ld)&o.medr.s.double_dd=median(double_dd)&o.misr.s.long_ld=missing(long_ld)&olap=true&o.sr.s.float_fd=sum(float_f)&o.mar.s.date_dtd=max(date_dtd)&o.ur.s.float_fd=unique(float_fd)&o.medr.s.int_id=median(int_id)&o.mar.s.long_ld=max(long_ld)&o.cr.s.double_dd=count(double_dd)&o.mir.s.date_dtd=min(date_dtd)&rows=0&o.sosr.s.int_id=sumofsquares(int_id)&o.mir.s.double_dd=min(double_dd)&o.sosr.s.double_dd=sumofsquares(double_dd)&o.p6r.s.long_ld=percentile(60,long_ld)&o.p2r.s.long_ld=percentile(20,long_ld)&o.cr.s.string_sd=count(string_sd)&o.str.s.long_ld=stddev(long_ld)&o.mar.s.double_dd=max(double_dd)&o.medr.s.float_fd=median(float_fd)&o.cr.s.int_id=count(int_id)&o.mir.s.string_sd=min(string_sd)&o.misr.s.float_fd=missing(float_fd)&o.sr.s.long_ld=sum(long_l)&o.mr.s.int_id=mean(int_id)&q=*:*&o.p6r.s.double_dd=percentile(60,double_dd)&o.mar.s.int_id=max(int_id)&o.mar.s.string_sd=max(string_sd)&o.sr.s.int_id=sum(int_i)&o.p2r.s.double_dd=percentile(20,double_dd)&o.ur.s.int_id=unique(int_id)&o.p2r.s.date_dtd=percentile(20,date_dtd)&o.mir.s.int_id=min(int_id)&o.cr.s.date_dtd=count(date_dtd)&o.p6r.s.date_dtd=percentile(60,date_dtd)}
hits=100 status=0 QTime=22
>> {code}
>>
>> But it's vastly different in the failed log:
>>
>> {code}
>> oasc.SolrCore.execute [collection1] webapp=null path=null params={o.medr.s.int_id=median(int_id)&o.p2r.s.string_sd=percentile(20,string_sd)&o.sr.s.long_ld=sum(long_l)&o.misr.s.string_sd=missing(string_sd)&o.mir.s.long_ld=min(long_ld)&o.ur.s.string_sd=unique(string_sd)&o.misr.s.float_fd=missing(float_fd)&o.p2r.s.date_dtd=percentile(20,date_dtd)&o.medr.s.date_dtd=median(date_dtd)&o.p2r.s.long_ld=percentile(20,long_ld)&o.cr.s.float_fd=count(float_fd)&o.p2r.s.double_dd=percentile(20,double_dd)&o.mar.s.int_id=max(int_id)&o.cr.s.long_ld=count(long_ld)&o.mr.s.int_id=mean(int_id)&o.medr.s.long_ld=median(long_ld)&o.p2r.s.int_id=percentile(20,int_id)&o.misr.s.int_id=missing(int_id)&o.p6r.s.date_dtd=percentile(60,date_dtd)&o.medr.s.double_dd=median(double_dd)&o.sosr.s.double_dd=sumofsquares(double_dd)&o.mir.s.int_id=min(int_id)&o.misr.s.long_ld=missing(long_ld)&o.cr.s.double_dd=count(double_dd)&o.str.s.double_dd=stddev(double_dd)&indent=true&o.p6r.s.long_ld=percentile(60,long_ld)&o.p6r.s.int_id=percentile(60,int_id)&o.st.s.float_fd=stddev(float_fd)&o.misr.s.date_dtd=missing(date_dtd)&rows=0&o.cr.s.string_sd=count(string_sd)&o.sr.s.int_id=sum(int_i)&o.ur.s.date_dtd=unique(date_dtd)&o.medr.s.float_fd=median(float_fd)&o.cr.s.date_dtd=count(date_dtd)&o.ur.s.long_ld=unique(long_ld)&o.misr.s.double_dd=missing(double_dd)&o.mar.s.string_sd=max(string_sd)&o.mir.s.double_dd=min(double_dd)&o.mir.s.date_dtd=min(date_dtd)&o.mir.s.string_sd=min(string_sd)&o.p6r.s.string_sd=percentile(60,string_sd)&o.mir.s.float_fd=min(float_fd)&o.sr.s.double_dd=sum(double_d)&o.str.s.int_id=stddev(int_id)&o.ur.s.float_fd=unique(float_fd)&o.cr.s.int_id=count(int_id)&o.str.s.long_ld=stddev(long_ld)&o.mar.s.long_ld=max(long_ld)&o.mr.s.long_ld=mean(long_ld)&o.mr.s.float_fd=mean(float_fd)&o.mar.s.date_dtd=max(date_dtd)&o.p6r.s.double_dd=percentile(60,double_dd)&o.sr.s.float_fd=sum(float_f)&o.ur.s.int_id=unique(int_id)&o.p6r.s.float_fd=percentile(60,float_fd)&o.ur.s.double_dd=unique(double_dd)&o.sosr.s.int_id=sumofsquares(int_id)&o.p2r.s.float_fd=percentile(20,float_fd)&q=*:*&o.sosr.s.long_ld=sumofsquares(long_ld)&o.mr.s.double_dd=mean(double_dd)&o.sosr.s.float_fd=sumofsquares(float_fd)&olap=true&o.mar.s.double_dd=max(double_dd)&o.mar.s.float_fd=max(float_fd)}
hits=100 status=0 QTime=164
>> {code}
>>
>> Don't know what may be the cause of this, sorry.
>>
>>
>>
>>> Fix up test failures for Analytics Component
>>> --------------------------------------------
>>>
>>>                 Key: SOLR-5488
>>>                 URL: https://issues.apache.org/jira/browse/SOLR-5488
>>>             Project: Solr
>>>          Issue Type: Bug
>>>    Affects Versions: 5.0, 4.7
>>>            Reporter: Erick Erickson
>>>            Assignee: Erick Erickson
>>>
>>> The analytics component has a few test failures, perhaps environment-dependent.
This is just to collect the test fixes in one place for convenience when we merge back into
4.x
>>
>>
>>
>> --
>> This message was sent by Atlassian JIRA
>> (v6.1#6144)
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message