axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran" <stev...@iseran.com>
Subject Re: Suggestion: Performance Anaylsis test
Date Tue, 03 Sep 2002 20:19:10 GMT

----- Original Message -----
From: "Matt Seibert" <mseibert@us.ibm.com>
To: <axis-dev@xml.apache.org>
Sent: Tuesday, September 03, 2002 11:56 AM
Subject: Suggestion: Performance Anaylsis test


> Hey guys,
>
> I've heard that there have been performance problems with AXIS in the
past,
> such as regressions from version to version in the throughput.  I would
> like to propose a test that simply analyzes the output from a test run to
> look a regressions in the time it takes for the tests to complete.

you need to distinguish network induced latencies from code perf, of course;
so localhost only tests, right?

The other issue is that there is more than just latency to test, the other
big thing is max load before latency goes through the roof; that takes more
time to run (but Sam is doing that kind of thing...)

> This, of course, would be optional, and would be dependant on setting some
> property.  The results would be stored in a flat file, of known format,
and
> stored in the test tree, so that you could plot the times out all by
> yourself.

an XML file, surely.

>
> How does this sound?  Anyone have an objection to doing this?  The main
> changes would be:
>       1) Creating the new test
>       2) Writing the test output lines to some given alternate file (like
> test/performance/<TEST_NAME.perf>
>       3) Comparing the old (most recent) test data with the new data
>       4) Determining a delta (if any)
>       5) Creating a ${test.reportdir}/TEST-test.report file which is an
> HTML view of this data
>             GREEN - Performance improvement
>             BLUE - performance the same
>             RED - performance degredation
>       6) Blowing away those intermediate test/performance/<TEST_NAME.perf>
> files
>       7) Committing the changed performance data file to CVS (if we want
to
> make this data public, static, and trackable)
>
> Okay, now I solicit input from you.  Let me know....

We need some tests that are useful for measuring latency

-echo System.currentTimeMillis() (I have a JNI to get Pentium clock ticks
for windows and linux for an alternate option; its one of the antbook sample
projects). This test would help calibrate round trip times.

-simple messages (string, int) and back
-simple message in, complex object back
-complex object in, simple object back
-complex both ways
-attachments
-same tests with various headers

Its good for bottleneck hunting if the response includes a breakdown of
time, by timestamping at different points in the message (receipt,
after-parse, after-process, after-marshall); lets you know where to focus
your energies.



Mime
View raw message