qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Conway <acon...@redhat.com>
Subject Re: Draft Interop Testing Spec - Please Read
Date Wed, 07 Mar 2007 15:44:10 GMT
Rupert Smith wrote:
> I don't think the test clients will need to use
> JUnit/CppUnit/WhateverUnit. Mostly they will just be fairly simple
> scripts that react to instructions sent by the coordinator. Although,
> having said that, you could use *Unit if you like. 
The test clients will need a way to:
 - group related tests together.
 - assert expected conditions are met.
 - handle unexpected failures.
 - collect and report results.
 - provide a natural test environment for each language

In other words the tests will need everything that the *Unit frameworks 
provide. It may not seem like a lot, but would you consider writing 
JUnit from scratch to test Java code? It won't be any easier to build a 
distributed test framework from scratch - in fact it'll be harder and 
the result will be less usable. Trust me, I've been there.

> I'm just not sure
> it adds anything, because these clients won't do anything when run on
> their own. They need the coordinator to push them along.
Right but how do you express that on the client side? I would say you 
express it as a "QpidTestRunner" test runner for *Unit tests. The 
QpidTestRunner is responsible for talking to the co-ordinator. The 
co-ordinator tells the test runner "Run BasicTests.getGet" the test 
runner sets up the test environment executes the corresponding JUnit 
test, collects the results and sends them back to the co-ordinator. The 
"test client" *is* a testrunner. That means we can flexibly load as many 
or as few tests as we want into the runner, run tests selectively and do 
all the other good stuff that you can do with *Unit tests. It also means 
the tests themselves are straight *Unit, nothing new to learn for test 

It may not seem like *Unit is doing a lot in the scenario above, but 
believe me, speaking as one who has participated in reinventing it 
several times *it is not worth reinventing*.

> The coordinator itself is a different story. I've sketched an outline
> for it in Java built on top of JUnit. Trying to use JUnits existing
> TestSuite mechanism, and ability to locate tests in *Test classes. A
> little bit of adaption perhaps needed to dynamically name the tests,
> or to run the same test case many times accross differnt combinations
> of clients. A root test case called something like InviteTestCase to
> provide common code to do the invite and gather reports, and provide
> convenience methods for common tasks etc. Here JUnit adds a lot as a
> convenient framework to base this on.
All the more reason to use it on both sides: give people a common 
framwork instead of making them jump from JUnit to some ad-hoc 
framework. E.g. with JUnit on both ends the client part of a test can 
sit right beside the co-ordinator part and use some simple naming 
conventions to make it obvious what goes with what. 
> I'll aim to get this skeleton code, and skeleton code for a java test
> client checked in before fleshing it out. That way, other test clients
> can copy the same structure.
Looking forward to it  :) I'll try to put some code where my mouth is 
when the skeleton is in place.


View raw message