lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Miller (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-4169) The test suite should consistently pass
Date Wed, 12 Dec 2012 16:33:22 GMT

    [ https://issues.apache.org/jira/browse/SOLR-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530080#comment-13530080
] 

Mark Miller edited comment on SOLR-4169 at 12/12/12 4:31 PM:
-------------------------------------------------------------

To add a bit of history:

Most of these fails never would have been fails when most of this code was written. Back then,
we didnt check for directories not being closed, we didnt run tests in parallel, tests ran
once a day, we didn't have an awesome test framework that failed based on all these things.

So before, most of what randomly fails now, would not even have failed before.

The original coders for a lot of the tests and code failing had moved on to other things.
It's taken time to slowly harden everything. SolrCloud and its jetty based tests introduced
a new layer.

Meanwhile, I do a lot of solr development and i find the tests very useful. If you need some
help, please, ask me and I can lend you hand. 

The tests have made great strides - I and a couple others are reducing the failures all the
time. Sometimes they flair up due to some change or mix of changes - currently that is happening
due to a couple issues that are well documented and attended to on the list. Real issues that
need to be dealt with.

The random fails are not usually test issues - but nor are they always critical issues. In
fact, most involve shutdown - an area we just have not cared a great deal about in Solr in
the past. So what if shutdown wasnt perfect. SolrCloud changes that a bit.

Anyhow, I put a lot of effort into improving the tests, and I know I've made a lot of ground,
and will continue to do so. There are a lot of challenges. Like I said, Ive been in the Lucene
world too, and the nature of it being a library without jetty or large integration tests really
made it an breeze in comparison. Lucene also is lucky enough to have more people that are
test fanatics at the moment.
                
      was (Author: markrmiller@gmail.com):
    For to add a bit of history:

Most of these fails never would have been fails when most of this code was written. Back then,
we didnt check for directories not being closed, we didnt run tests in parallel, tests ran
once a day, we didn't have an awesome test framework that failed based on all these things.

So before, most of what randomly fails now, would not even have failed before.

The original coders for a lot of the tests and code failing had moved on to other things.
It's taken time to slowly harden everything. SolrCloud and its jetty based tests introduced
a new layer.

Meanwhile, I do a lot of solr development and i find the tests very useful. If you need some
help, please, ask me and I can lend you hand. 

The tests have made great strides - I and a couple others are reducing the failures all the
time. Sometimes they flair up due to some change or mix of changes - currently that is happening
due to a couple issues that are well documented and attended to on the list. Real issues that
need to be dealt with.

The random fails are not usually test issues - but nor are they always critical issues. In
fact, most involve shutdown - an area we just have not cared a great deal about in Solr in
the past. So what if shutdown wasnt perfect. SolrCloud changes that a bit.

Anyhow, I put a lot of effort into improving the tests, and I know I've made a lot of ground,
and will continue to do so. There are a lot of challenges. Like I said, Ive been in the Lucene
world too, and the nature of it being a library without jetty or large integration tests really
made it an breeze in comparison. Lucene also is lucky enough to have more people that are
test fanatics at the moment.
                  
> The test suite should consistently pass
> ---------------------------------------
>
>                 Key: SOLR-4169
>                 URL: https://issues.apache.org/jira/browse/SOLR-4169
>             Project: Solr
>          Issue Type: Bug
>          Components: Tests
>            Reporter: Robert Muir
>            Priority: Blocker
>             Fix For: 4.1
>
>
> I'm at the end of my rope. 
> Last week I tried to help with this: but its already wednesday and solr tests are 20x
worse.
> Users are complaining that they cannot get snapshots and test them, because the tests
fail all the time.
> Maven dependencies are wrong, but we never knew, because the tests never succeeded so
the validation task hasn't actually run in months.
> I think fixing this situation is a blocker to release: I can't imagine a bigger blocker
actually.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message