tapestry-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Gidley <...@gidley.co.uk>
Subject Re: Performance Testing Lab
Date Fri, 03 Sep 2010 05:55:11 GMT
I did a comparative load test
http://blog.gidley.co.uk/2009/05/tapestry-load-testing-round-up.html a while
back and we do a lot of load testing for SeeSaw.com (mainly to compare
versions). I did create a 'comparative' struts application for that
evaluation (source code
http://sites.google.com/a/gidley.co.uk/tapestryloadtest/Home/test-5)

We use Grinder as our load test tool - as it handles clustered load
generating agents well and the Jython scripting makes it easy to share steps
between 'journeys' in the application. You probably don't need to worry
about clustering - as it isn't a core tapestry 'feature' (rather an
environmental one), but even without that feature grinder is easy to work
and powerful.

If your goal is to test tapestry the first issue you will hit is any Data
Access - in our apps the data access piece is 90% of the page time. In
SeeSaw we 'solve' this by heavily caching data so most pages only access
data from a Cache. I would suggest to get a test that focused on tapestry
you load you data from in memory caches. That way you won't be measuring how
fast Hibernate or your DB(etc) performs.

Another big issue we hit is breaking/maintaining our load test scripts - as
they effectively reverse engineer some tapestry behaviours (especially AJAX
ones) they can get broken quite easily. Our solution is to run them single
threaded as part of our CI builds. This usually catches obvious changes.

Another thing to watch out for is your app server. Jetty is an excellent
high load container - but depending on how it is configured it can
'throttle' throughput (this is excellent thing in production). This can skew
your test as you end up measuring the throttling and not the application.
The trick is to set the thread pools in Jetty far higher than you would in
production so you can be sure you are not maximising them.

I would advise trying to use controlled hardware for your test - slight
environmental differences can totally adjust the results. EC2 and its like
are popular for this - do Apache have anything similar? I can get you time
on (but not direct access to) the ioko cloud (which is a VMWare VSphere
cluster), which does let you nearly guarantee QOS.

It may be a good idea to build the test as something that can be easily
deployed as a Virtual machine as it would let you easily deploy it anywhere.
I have previously used Suse Studio (http://susestudio.com/) to build a very
small VM with a maven script on it that updates/runs my app. The VM's built
there will run pretty much anywhere.

To find issues with a load test I recommend YourKit. It gives you a ton of
monitoring and via its 'probes' you can see right inside tapestry. It is
free for open source projects. I would also recommend enabling JMX in your
application and periodically writing the stats somewhere. You can do this
simply with VisualVM or you can get more advanced and store them in a
database and graph them - ioko use Cacti http://www.cacti.net/ and
http://jmxmonitor.sourceforge.net/ (though it may be a bit more complex for
what you want).


Ben Gidley

On Thu, Sep 2, 2010 at 11:07 PM, Thiago H. de Paula Figueiredo <
thiagohp@gmail.com> wrote:

> On Thu, 02 Sep 2010 18:30:49 -0300, Howard Lewis Ship <hlship@gmail.com>
> wrote:
>
>  One of my long term and unrealized goals for Tapestry is to have a
>> legitimate performance testing lab.
>>
>
> Cool!
>
>
>  I'd like the ability to run a "standard" application through its
>> paces, collect statistics, and see how well each new Tapestry release
>> compares to its predecessor in terms of general performance: response
>> time, memory utilization, saturation and recovery.  Ideally, it would
>> be nice to create equivalent JSP/Spring MVC/Struts/Wicket
>> implementations of the same app.
>>
>
> I've done something a little similar, but in a smaller scope, for a
> presentation comparing Struts 1, Struts 2, JSF and Tapestry 5. I implemented
> a simple application, DAOs and simple business rules layer, and them
> implemented its web interface using each of the frameworks.
>
>
>  I'm really concerned with ensuring that 5.2 out-performs 5.1, and uses
>> less memory.  The singleton pages approach should lower the memory
>> utilization ... but I'm concerned that the new AOP stuff inside
>> ClassTransformation (which has a tendency to create lots of little
>> one-off classes) will overshadow the other improvements.
>>
>
> Someone in the mailing list mentioned Parfait (
> http://code.google.com/p/parfait/wiki/IntroductionToParfait). It could be
> helpful for that.
>
> --
> Thiago H. de Paula Figueiredo
> Independent Java, Apache Tapestry 5 and Hibernate consultant, developer,
> and instructor
> Owner, Ars Machina Tecnologia da Informação Ltda.
> http://www.arsmachina.com.br
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: dev-help@tapestry.apache.org
>
>


-- 
Ben Gidley

www.gidley.co.uk
ben@gidley.co.uk

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message