lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-10807) Randomize PointFields in all tests unless explicit reason not to
Date Thu, 20 Jul 2017 01:14:00 GMT

    [ https://issues.apache.org/jira/browse/SOLR-10807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16094046#comment-16094046
] 

ASF subversion and git services commented on SOLR-10807:
--------------------------------------------------------

Commit 25c1ac7423a301e05b53cee2f0f022756c6c5c5e in lucene-solr's branch refs/heads/branch_7_0
from Chris Hostetter
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=25c1ac7 ]

SOLR-10807: CHANGES simplification to summarize all completed sub-tasks

(cherry picked from commit 82d29e3a091476cf27ecadc2a32fca4bea50f13c)


>  Randomize PointFields in all tests unless explicit reason not to
> -----------------------------------------------------------------
>
>                 Key: SOLR-10807
>                 URL: https://issues.apache.org/jira/browse/SOLR-10807
>             Project: Solr
>          Issue Type: Task
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Hoss Man
>            Assignee: Hoss Man
>            Priority: Blocker
>             Fix For: 7.0, master (8.0), 7.1
>
>         Attachments: core.test.log.fde06f34b7f9d0916a134b3efaa8780892ff8e39.txt, core.test.log.txt
>
>
> We need to seriously beef up our testing of PointFields to figure out what Solr features
don't currently work with PointFields.
> The existing Trie/Point randomization logic in SolrTestCaseJ4 is a good start -- but
only a handful of schema files leverage it.
> ----
> Allthough a jira/SOLR-10807 branch was originally created with this goal, it was ultimately
just used for initial experimentation, and has been abandoned.  The "meat" of the work needed
to improve _how_ we randomize in Point fields was done in SOLR-10864, and other sub-tasks
of this issue have been / are being used to track rolling out this randomization to more and
more test schema files and validating the affected tests.
> This effort is now highly parallelizable  -- so here are some rough guidelines/suggestions
for folks interested in contributing to this effort:
> * create a subtask identifying the name (or glob) of the test-files schema file you plan
to tackle before starting (so multiple people don't duplicate work on the same tests
> * run the following one liner (assumming bash/perl) to change all Trie field types in
the schema(s) to use the new randomized system vars...{code}
> find -name \*your-schema-glob-or-name\* -type f | xargs perl -i -ple 's/class="solr.TrieIntField"/class="solr.TrieIntegerField"/g;
s/class="solr.Trie(.*)Field"/class="\${solr.tests.$1FieldType}"/g; unless (/docValues/) {
s/(class="\${solr.tests..*FieldType}")/$1 docValues="\${solr.tests.numeric.dv}"/g; }'
> {code}
> * identify the affected tests
> ** grep for the schema file names in all test classes to start building the list
> ** recursively check each test class in the list for subclasses
> * hammer on all affected tests with many diff seeds
> ** NOTE: you can force the points vs trie choice by specifying {{-Dsolr.tests.use.numeric.points=true}}
vs {{-Dsolr.tests.use.numeric.points=false}}
> *** folks with beefy machines may find it handy to use 2 git working dirs to hammer on
diff seeds with a diff hardcoded values of that sysprop
> * If you encounter any test failures...
> ** figure out the root  cause
> ** file a new "Bug" jira, link it as related to SOLR-10807 & SOLR-8396
> *** NOTE: first double check there isn't already a bug on point linked to from one of
those places
> ** use {{@SuppressPointFields}} (citing the new jira) if necessary for any functionality
that absolutely will not work with point fields
> ** use something like this in tests where functionality requires docValues in order to
work properly with points (although in practice, the comment should always cite the relevant
jira)
> *** https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;a=blobdiff;f=solr/core/src/test/org/apache/solr/handler/component/DistributedFacetPivotLargeTest.java;h=eb6f54d5db42ba6f24d88d9bbda78bfab198a94e;hp=0c5e128ba8a5cf1a8b488b17ddf9ce2030e0f22d;hb=38f29b2;hpb=a948e1714609ef662184c71eedb219caf44fc037
> ** use something like this if a small subset of a test is known to not work with points...
> *** https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;a=blobdiff;f=solr/core/src/test/org/apache/solr/search/function/TestFunctionQuery.java;h=4cee94b86bb41dcc451f72c81c21f6ed911b9e95;hp=f19e4b08530c9650515c64ef7d589f21db939ccf;hb=38f29b2;hpb=a948e1714609ef662184c71eedb219caf44fc037
> ** use something like this if the test has a need/reason to care/assert what the underying
FieldType is of a numeric field...
> *** https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;a=blobdiff;f=solr/core/src/test/org/apache/solr/schema/IndexSchemaTest.java;h=4719f0408fbc39e8b806197c2eb0b26bf533b0c7;hp=7790859fcde43fb9ecedaed8a146aef3cac1ae10;hb=38f29b2;hpb=a948e1714609ef662184c71eedb219caf44fc037
> * when committing changes, the commit msg should cite both the original sub-task jira#,
as well as any bug jira#s that needed special annotation/handling/assumptions - so that in
the future people working on fixing those bugs have easy to find GIT SHAs identifying when/where
tests are currently hacked to avoid the bugs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message