lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cassandra Targett (JIRA)" <>
Subject [jira] [Commented] (SOLR-10032) Create report to assess Solr test quality at a commit point.
Date Wed, 25 Jan 2017 15:41:27 GMT


Cassandra Targett commented on SOLR-10032:

+1. Flakey test clean up is an arduous and unforgiving task, but it's hard to overstate it's

Do you have ideas on how we might track this beyond trying to fix the biggest offenders? If
there's stuff we can do to raise the visibility of such tests, that will be a start to getting
all of us more aware of the problem and moving toward a long-term solution.

> Create report to assess Solr test quality at a commit point.
> ------------------------------------------------------------
>                 Key: SOLR-10032
>                 URL:
>             Project: Solr
>          Issue Type: Task
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: Tests
>            Reporter: Mark Miller
>            Assignee: Mark Miller
> We have many Jenkins instances blasting tests, some official, some policeman, I and others
have or had their own, and the email trail proves the power of the Jenkins cluster to find
test fails.
> However, I still have a very hard time with some basic questions:
> what tests are flakey right now? which test fails actually affect devs most? did I break
it? was that test already flakey? is that test still flakey? what are our worst tests right
now? is that test getting better or worse?
> We really need a way to see exactly what tests are the problem, not because of OS or
environmental issues, but more basic test quality issues. Which tests are flakey and how flakey
are they at any point in time.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message