qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Wall <keith.w...@gmail.com>
Subject Re: Ideas to rationalise the Java test profiles.
Date Tue, 05 Jul 2011 21:22:11 GMT
Hi Rajith

> 1. Coverage
>  We currently have close to 1200+  test cases and my impression is that we
still have quite a few gaps. All though it's painful maybe we should try to
get an inventory of our existing test cases.

I think the best form of inventory would be the tests themselves. The test
methods need clear meaningful names and, if the test has unavoidably
complexity, supporting method level Javadoc.     I'd worry that a separate
test inventory would get quickly out of date.

I agree we have many gaps. I'd like to get coverage reports turned on
Jenkins, perhaps on a nightly schedule.  I realise coverage alone is no
silver bullet, but it can be a useful pointer to areas of weakness.

> 2. Manageability/Maintainability of test suites  As Robbie pointed out we
have a lot of low utility tests. these tests just bloat our test suite but
add very little value.  We need to get out of the habit of adding a new test
case for every commit. I personally think we have too many tests.

I'd say our main problem is an over-reliance on system tests. We have cases
where we have exhaustively exercised _every_ feature of a use-case from a
system test.   This approach makes for a slow test suite that offers the
developer little support when refactoring (the test failure is often to
granular to be useful).   I think a better approach is to rely on unit
test(s) to drive out of the behaviour, then back these up with a smaller
number of judicious system tests.

> 3. System tests    We need to make a distinction between client tests and
broker tests

Yes, agree. I'm suggesting we add a Java Testing Howto to the website.  This
can help make the distinction between unit/system tests and help our
contributors submit useful test cases with their patches.

> and a developer should have the flexibility of running either or both.

I think we already have this.  The -Dtest=xx  operand can be an Ant glob.

For instance:
ant -f build.xml test -Dtest='org/apache/qpid/server/**/*'
will run all the tests in package org.apache.qpid.server downwards,
regardless of depth
Of course, if the package structure in the test tree made sense, it would be
even more useful.

kind regards Keith.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message