qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robbie Gemmell (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (QPIDJMS-75) make the test peer report more useful failure causes and update related tests to be more forgiving of slowdowns on shared CI systems
Date Fri, 14 Aug 2015 10:40:46 GMT

     [ https://issues.apache.org/jira/browse/QPIDJMS-75?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Robbie Gemmell updated QPIDJMS-75:
    Issue Type: Test  (was: Task)

> make the test peer report more useful failure causes and update related tests to be more
forgiving of slowdowns on shared CI systems
> ------------------------------------------------------------------------------------------------------------------------------------
>                 Key: QPIDJMS-75
>                 URL: https://issues.apache.org/jira/browse/QPIDJMS-75
>             Project: Qpid JMS
>          Issue Type: Test
>    Affects Versions: 0.1.0, 0.2.0, 0.3.0
>            Reporter: Robbie Gemmell
>            Assignee: Robbie Gemmell
>             Fix For: 0.4.0
> In adddition to running unit tests and some brokered interop/integration tests, we also
created a 'test peer' to allow performing some brokerless integration testing of the client
against a peer we could use to match/validate the AMQP frames emitted by the client and construct
AMQP frames to send to the client.
> Thus far, whenever a particular matcher fails (e.g checking  a particular frame field
has an expected value, and it did not), the resulting assertion error in the test peer thread
was recorded and then peer simply stops processing, wihtout undertaking any action that would
have occurred had the value been as expected. As a result, if the client is awaiting a response
from the peer (which in most cases it is) then it is never sent, and the test idles until
it is timed out by JUnit (or ceased by some other action), which has also resulted in use
of artificially low timesouts to bound run times against 'expected' failure cases, additionally
making the tests unforgiving of sporadic slowdown on shared CI machines. Although the assertion
failure is recorded, the test peer typically is not shut down in those cases as the test timeout
itself becomes the reported cause of failure, leading to inspection of the test log output
being necessary to identify anything about what actually happened.
> To improve things in terms of the reported test failure/error cause and also the time
taken for the test to fail/error in many cases, assertion failures continue to be recorded
but the subsequent actions actually still performed. Where the test is able to continue to
completion the first assertion can then be thrown when closing down the test peer, meaning
the test is much more likely to fail fast and the assertion failure actually becoming the
reported cause/error by JUnit on the console, thus improving reporting and simplifying analysis.
By avoiding need to use of the test timeout to bound run time in the case of these 'expected
failures', the applied test timeouts can also be increased which enables the tests to be more
forgiving of sporadic slowdowns while being run on shared CI machines.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org

View raw message