james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bernd Fondermann" <bernd.fonderm...@googlemail.com>
Subject Re: Milestone from trunk
Date Sun, 04 May 2008 12:47:52 GMT
On Sat, May 3, 2008 at 1:43 PM, Noel J. Bergman <noel@devtech.com> wrote:
> Bernd Fondermann wrote:
>  > Noel J. Bergman wrote:
>
> > > The proposal is based on the fact that every message delivered to the
>  zone
>  > > will be disposable spam.  Therefore, unlike performing some sort of faux
>  > > release without any basis, we will be testing in a risk-free
>  environment.
>  > > Every message can be dropped, the database can be corrupted, the server
>  can
>  > > leak memory and crash, and no one should care other than to fix it.
>
>  > Then you are talking about a closed environment test, so do I.
>
>  We're possibly differing on some English language semantics in terminology.
>  As long as we agree with the mechanics of the proposal, as above, I don't
>  care if you call it Nancy.  :-)
>
>
>  > > As an example of why this sort of testing is the right thing to do,
>  rather
>  > > than the idiocy of pushing out releases without real-world testing,
>  consider
>  > > our current situation with SVN.
>
>  > I never said we should release without testing beforehand.
>  > It is you who is ignoring the fact we have a safe black box
>  > testing tool right here in our project.
>
>  No, you appear to be ignoring that blackbox testing using Postage in
>  isolated networks is nowhere near sufficient to provide any basis for
>  assurance of real-world functionality.  I am addressing that lack.
>
>
>  > At the bottom line, I am happy that there seems to emerge some kind of
>  > common goal to start from TRUNK and put it into heavy testing, be it on
>  > a Solaris zone or locally using Postage.
>
>  See above regarding Postage.  And keep in mind that I've used Postal and
>  Rabid for isolated testing in the past, but it (too) is not sufficient.  We
>  need the real world exposure.

Every test setup has its pros and cons. Unit test have a specific use,
so have isolated, reproducible functional or load tests.

Nancy (solaris zone, that is ;-)) has the disadvantage that it
probably will reveil some bugs/problems which are not (easily)
reproducible because we don't control the test data/load. This would
make some people with proper knowledge of testing not call it a
'test', more an  'experiment'. I think it's worthwhile anyway, but
you'd have to fall back to other tests after running into that certain
family of seldom problems.

I seem to have a hard time pitching Postage (which I actually wrote
after - or better because of - working with Postal and Rabid), which
allows to use specialized message factory objects. Write a custom
factory for every specific mail which leads to a specific false
behavior inside the Server. This makes Postage a fit for any kind a
functional test you might (have to) come up with.

  Bernd

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message