qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weston M. Price" <wpr...@redhat.com>
Subject Re: Ideas to rationalise the Java test profiles.
Date Fri, 08 Jul 2011 13:38:31 GMT

On Jul 5, 2011, at 5:22 PM, Keith Wall wrote:

> Hi Rajith
>> 1. Coverage
>> We currently have close to 1200+  test cases and my impression is that we
> still have quite a few gaps. All though it's painful maybe we should try to
> get an inventory of our existing test cases.
I have started work on this as one of my tasks and would be more than happy to work with someone.

> I think the best form of inventory would be the tests themselves. The test
> methods need clear meaningful names and, if the test has unavoidably
> complexity, supporting method level Javadoc.     I'd worry that a separate
> test inventory would get quickly out of date.
> I agree we have many gaps. I'd like to get coverage reports turned on
> Jenkins, perhaps on a nightly schedule.  I realise coverage alone is no
> silver bullet, but it can be a useful pointer to areas of weakness.
+1 to both comments.

>> 2. Manageability/Maintainability of test suites  As Robbie pointed out we
> have a lot of low utility tests. these tests just bloat our test suite but
> add very little value.  We need to get out of the habit of adding a new test
> case for every commit. I personally think we have too many tests.
> I'd say our main problem is an over-reliance on system tests. We have cases
> where we have exhaustively exercised _every_ feature of a use-case from a
> system test.   This approach makes for a slow test suite that offers the
> developer little support when refactoring (the test failure is often to
> granular to be useful).   I think a better approach is to rely on unit
> test(s) to drive out of the behaviour, then back these up with a smaller
> number of judicious system tests.

>> 3. System tests    We need to make a distinction between client tests and
> broker tests
> Yes, agree. I'm suggesting we add a Java Testing Howto to the website.  This
> can help make the distinction between unit/system tests and help our
> contributors submit useful test cases with their patches.
>> and a developer should have the flexibility of running either or both.
> I think we already have this.  The -Dtest=xx  operand can be an Ant glob.
> For instance:
> ant -f build.xml test -Dtest='org/apache/qpid/server/**/*'
> will run all the tests in package org.apache.qpid.server downwards,
> regardless of depth
> Of course, if the package structure in the test tree made sense, it would be
> even more useful.
> kind regards Keith.

Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org

View raw message