maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kristian Rosenvold (JIRA)" <j...@codehaus.org>
Subject [jira] Updated: (SUREFIRE-611) surefire exit code should explicitly distinguish between crashed and failed unit test(s)
Date Fri, 24 Dec 2010 20:30:58 GMT

     [ http://jira.codehaus.org/browse/SUREFIRE-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kristian Rosenvold updated SUREFIRE-611:
----------------------------------------

    Component/s: Maven Surefire Plugin

> surefire exit code should explicitly distinguish between crashed and failed unit test(s)
> ----------------------------------------------------------------------------------------
>
>                 Key: SUREFIRE-611
>                 URL: http://jira.codehaus.org/browse/SUREFIRE-611
>             Project: Maven Surefire
>          Issue Type: Improvement
>          Components: Maven Surefire Plugin
>            Reporter: Tycho Lamerigts
>         Attachments: tc-junit-crash.tar.gz
>
>
> - If a unit test crashes, surefire skips all remaining tests, prints a message "[ERROR]
There are test failures.  ..." to the console, and does NOT print a test summary report.
> - If one or more unit tests fail, maven completes remaining tests, prints a message "[ERROR]
There are test failures.  ..." to the console, and DOES print a test summary report.
> The only distinction between crash and fail is the presence (or absence) of a summary
report.
> This is a problem for some tools (like TeamCity), because they typically don't pick up
on the fact that many testcases were not run at all in case of a crashed unit test. Instead
in case of a crashed test they simply report, for example, 100 tests succeeded, 0 failed.
They do not report "50 tests skipped due to a crash after test #100", and it is quite hard
for those tools to recognize this situation. If surefire would report a crash more explicitly,
other tools can act accordingly.
> I have attached a small maven project to reproduce the problem. By commenting and uncommenting
the marked line in file ATest.java, you can reproduce both fail and crash scenarios. In the
crash scenario, Maven will correctly report a failed build (unless you use the -Dmaven.test.failure.ignore
flag, in which case it even won't do that), but it fails to indicate there was a crash which
caused tests (in this case 3) to be skipped.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message