maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] Tibor17 commented on issue #221: [SUREFIRE-1638] Support running tests until failure or N test runs
Date Thu, 14 Feb 2019 13:38:22 GMT
Tibor17 commented on issue #221: [SUREFIRE-1638] Support running tests until failure or N test
runs
URL: https://github.com/apache/maven-surefire/pull/221#issuecomment-463629551
 
 
   I think you are talking about existing feature skipAfterFailureCaount.
   You should use it together with reRun feature which exists too.
   
   On Wed, Feb 13, 2019 at 6:03 PM Galder Zamarreño <notifications@github.com>
   wrote:
   
   > This is an initial proposal to add surefire/failsafe property called
   > untilFailureLoopCount.
   >
   > I often find myself in the situation where there's a random test failure
   > happening in CI, and have to run the one or several tests continuously
   > until it fails, maybe running them with TRACE logging.
   >
   > Normally, I'd wrap the mvn test... call with a script that calls it in a
   > loop, but this can be quite "slow" since each iteration is a new JVM
   > launch...etc.
   >
   > The idea behind untilFailureLoopCount is that you've give it a positive
   > number, say 1000, and the test(s) would run continuously until either a
   > failure happens or the test is executed that many times.
   >
   > I've created this randomly failing test
   > <https://github.com/galderz/surefire-until-failure-loop-count/blob/master/src/test/java/com/acme/maven/surefire/untilfailure/UntilFailureLoopCountTest.java>
   > and I've run it with mvn -Dsurefire.untilFailureLoopCount=10 test and the
   > output would be something like:
   >
   > [INFO] -------------------------------------------------------
   > [INFO]  T E S T S
   > [INFO] -------------------------------------------------------
   > [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   > [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.034 s - in
com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   > [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   > [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in
com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   > [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   > [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   > [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   > [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   > [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   > [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.001 s <<<
FAILURE! - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
   > [ERROR] com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest.testEventualFailure
 Time elapsed: 0.001 s  <<< FAILURE!
   > java.lang.AssertionError: Just messing with your testing
   > 	at com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest.testEventualFailure(UntilFailureLoopCountTest.java:16)
   >
   > If maintainers are interested in this, I can complete the PR with tests,
   > documentation and support for other providers.
   > ------------------------------
   > You can view, comment on, or merge this pull request online at:
   >
   >   https://github.com/apache/maven-surefire/pull/221
   > Commit Summary
   >
   >    - Support running tests until failure or N test runs
   >
   > File Changes
   >
   >    - *M*
   >    maven-failsafe-plugin/src/main/java/org/apache/maven/plugin/failsafe/IntegrationTestMojo.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-0> (21)
   >    - *M*
   >    maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/AbstractSurefireMojo.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-1> (7)
   >    - *M*
   >    maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/booterclient/BooterSerializer.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-2> (3)
   >    - *M*
   >    maven-surefire-common/src/test/java/org/apache/maven/plugin/surefire/AbstractSurefireMojoJava7PlusTest.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-3> (11)
   >    - *M*
   >    maven-surefire-common/src/test/java/org/apache/maven/plugin/surefire/AbstractSurefireMojoTest.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-4> (11)
   >    - *M*
   >    maven-surefire-common/src/test/java/org/apache/maven/plugin/surefire/MojoMocklessTest.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-5> (11)
   >    - *M*
   >    maven-surefire-common/src/test/java/org/apache/maven/plugin/surefire/booterclient/BooterDeserializerProviderConfigurationTest.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-6> (5)
   >    - *M*
   >    maven-surefire-plugin/src/main/java/org/apache/maven/plugin/surefire/SurefirePlugin.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-7> (21)
   >    - *M*
   >    surefire-api/src/main/java/org/apache/maven/surefire/booter/SurefireReflector.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-8> (3)
   >    - *M*
   >    surefire-api/src/main/java/org/apache/maven/surefire/testset/TestRequest.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-9> (18)
   >    - *M*
   >    surefire-booter/src/main/java/org/apache/maven/surefire/booter/BooterConstants.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-10> (1)
   >    - *M*
   >    surefire-booter/src/main/java/org/apache/maven/surefire/booter/BooterDeserializer.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-11> (3)
   >    - *M*
   >    surefire-providers/surefire-junit4/src/main/java/org/apache/maven/surefire/junit4/JUnit4Provider.java
   >    <https://github.com/apache/maven-surefire/pull/221/files#diff-12> (10)
   >
   > Patch Links:
   >
   >    - https://github.com/apache/maven-surefire/pull/221.patch
   >    - https://github.com/apache/maven-surefire/pull/221.diff
   >
   > —
   > You are receiving this because you are subscribed to this thread.
   > Reply to this email directly, view it on GitHub
   > <https://github.com/apache/maven-surefire/pull/221>, or mute the thread
   > <https://github.com/notifications/unsubscribe-auth/AA_yR6pCCw8XbpodQl7SNTkhII_HFUDKks5vNEVHgaJpZM4a52vY>
   > .
   >
   
   
   -- 
   Cheers
   Tibor
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message