maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Kabashnyuk (JIRA)" <j...@codehaus.org>
Subject [jira] (SUREFIRE-1044) Changed behaviour of TestNG test execution after implementing runOrder support
Date Wed, 04 Dec 2013 20:34:45 GMT

    [ https://jira.codehaus.org/browse/SUREFIRE-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=336783#comment-336783
] 

Sergey Kabashnyuk commented on SUREFIRE-1044:
---------------------------------------------

I made some research about what we have now in surefire plugin 2.16
Basically we have one testng sute for each test class.

But after reading testng documentation


http://testng.org/doc/documentation-main.html
A suite is represented by one XML file. It can contain one or more tests and is defined by
the <suite> tag.

And reviewing author of TestNG plugin to eclipse there he use only one sute per run as it
was in surefire plugin 2.15
https://github.com/cbeust/testng-eclipse/blob/master/src/main/org/testng/eclipse/refactoring/TestNGXmlPage.java

Make me feel that old (2.15) behaviour was "more" correct :)



I've made my fork of maven-surefire and add several changes
https://github.com/skabashnyuk/maven-surefire/commit/35422dd1901931608089b5803bfeb12dccc13d68
CheckTestNgVersionsIT.test685  - is working. Other test fail.

But to approve this strategy we should consider such facts:

Fact 1: Support of run order from 6.1.  TESTNG-476: <test> tags are now run in the order
found in testng.xml
http://beust.com/weblog/2011/07/01/announcing-testng-6-1/
https://github.com/cbeust/testng/commit/24215389671eeb04764ee673748987ef2c0fb367

Fact 2: Made preserve-order true by default.
https://github.com/cbeust/testng/commit/8849b3406ef2184ceb6002768a2d087d7a8de8d5


So, doing this way we may preserv test order wit testNg versions >6.1-6.3

I thought if we REALY need support for older version TestNGExecutor may play with group or
method dependencies
(http://testng.org/doc/documentation-main.html#dependent-methods)




                
> Changed behaviour of TestNG test execution after implementing runOrder support
> ------------------------------------------------------------------------------
>
>                 Key: SUREFIRE-1044
>                 URL: https://jira.codehaus.org/browse/SUREFIRE-1044
>             Project: Maven Surefire
>          Issue Type: Bug
>          Components: TestNG support
>    Affects Versions: 2.16
>            Reporter: Sergey Kabashnyuk
>
> I have some code automation around TestNG listeners. And I notice different behaviour
of maven-surefire-plugin 2.15 and 2.16.
> In 2.15  In method  org.testng.TestNG.run during execution contains 1-XmlTestSuite with
1-XmlTest with 2 XmlClass inside. 
> I've also checked InteliJ org.testng.RemoteTestNG he has the same state during execution.
> But in 2.16 we have 2-XmlTestSuite with 1-XmlTest with 1 XmlClass inside. 
> As a result TestNG initialise two IInvokedMethodListener instead of one. 
> Is this expected behaviour?
> Example of test code I put here 
> https://github.com/skabashnyuk/mvntestng

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message