jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheen-Pin Lim <cheenpin....@gmail.com>
Subject Re: Web Driver based Sampler?
Date Sun, 22 Apr 2012 02:27:04 GMT
Hi Chaitanya,

I guess I am looking at this from a browser view rather than from the
server side.  What I tend to find is that current load testing does not
properly reflect the browser load times on our site.  The reports from
Google (in webmasters tools) and newrelic.com give a very different picture
from that done during load/performance testing.

We use HP Load Runner for our in-house load testing and the numbers we get
(even when we download 'all' external resources) does not come close to the
numbers reported by the other tools.  I guess I am attempting to address a
gap that I see in our in-house load/performance testing, and the
integration of webdriver into JMeter could solve this problem.  At this
stage it is an attempt to try to provide a means to simulate what our
production servers see (i.e. using a real browser that will spend time
executing JS/CSS).

Yes, I agree that the complexity and cost of running a browser instance is
higher than using the standard HTTP Sampler, but with modern services such
as Amazon AWS, I would think that these problems would be
addressable/solvable.

regards,
CP

On 22 April 2012 06:30, chaitanya bhatt <bhatt.chaitanya@gmail.com> wrote:

> *On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim <cheenpin.lim@gmail.com
> >wrote:
> >Once you have more load
> >applied to the servers/services different characteristics tend to emerge,
> >which is why I was thinking that the use of JMeter would be useful.
>
> *I may totally sound cynical while saying this but honestly the reason why
> I am criticising your idea is mainly due to the week rational behind your
> motivation to create this feature:
>
> 1. First of all browser induced latency on the total workload is grossly
> overrated. Having said that, all that an accurate simulation of browser
> latency would do to your workload is that it would marginally reduce the
> total hits made on the server, which I think is bad from a fundamental load
> testing perspective.
> 2.To accommodate in-memory containers to hold 100s-1000s of virutal user
> browser sessions is practically impossible without testers having to
> significantly size up their test lab resources. Think about the cost
> factor.
> 3. Looking at todays trend of browser adoption your web driver would have
> to be equipped with simulation capability multiple types of browsers such
> as IE, Chromer, Safari etc. etc. and also you would have to consider
> simulating various device characteristics on the generated load. Think
> about the complexity.
>
> @Shmuel: The loadrunner protocol you are talking about is Ajax
> TrueClient. HP came out with Ajax TrueClient not with an intention
> of generating real browser traffic but it was mainly developed to reduce
> the complexity of scripting apps that employ unintelligible data
> serialization like Google GWT. With this HP's protocol you wouldn't have a
> necessity of correlating session variables at all and there by you would be
> significantly reducing the time to develop scripts. However, this protocol
> currently has poor user adoption due to various limitations.
>
> Thanks
> Chaitanya M Bhatt
> http://www.performancecompetence.com
>
> On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim <cheenpin.lim@gmail.com
> >wrote:
>
> > Hi,
> >
> > The tools mentioned help a great deal, however they generally look at it
> > from a single browser to a single server scenario.  Once you have more
> load
> > applied to the servers/services different characteristics tend to emerge,
> > which is why I was thinking that the use of JMeter would be useful.
> >
> > Furthermore, if our primary audience is hitting the site using a browser,
> > then would it not make sense to use the same tool to simulate the load?
> >
> > Just my thoughts.
> >
> > regards,
> > CP
> >
> > On 21 April 2012 17:26, chaitanya bhatt <bhatt.chaitanya@gmail.com>
> wrote:
> >
> > > Pleanty of awesome tools are already available for free to meet your
> > goals.
> > >
> > > Free:
> > > https://developers.google.com/pagespeed/
> > > http://yslow.org/
> > >
> > > Free or Premium:
> > > http://ajax.dynatrace.com/ajax/en/content/c-speed-page-load.aspx
> > >
> > >
> > > Selenium has a webdriver which you can plan on extending - if you will.
> > > IMHO Jmeter is not the right platform for this.
> > > Thanks
> > > Chaitnaya M Bhatt
> > > http://www.performancecompetence.com
> > > On Fri, Apr 20, 2012 at 11:55 PM, Cheen-Pin Lim <
> cheenpin.lim@gmail.com
> > > >wrote:
> > >
> > > > Hi,
> > > >
> > > > After looking at the current HTTP Sampler, I would like to propose a
> > > > WebDriver based implementation.  The problem is that with a lot of
> > modern
> > > > websites using AJAX, advanced CSS3 (transitions/transforms) to do the
> > > > rendering, the performance problem may not be 'visible' to the
> standard
> > > > HTTP Sampler.  The idea is that if we could use JMeter to control a
> > > browser
> > > > and collect the time it takes to render each 'page' it would give the
> > > users
> > > > an understanding of how long a page would take to render.
> > > >
> > > > Some of the high level goals are as follows:
> > > >
> > > >   1. Allow any WebDriver supported browser to be used to perform the
> > > >   Sampling.
> > > >   2. Provide a BeanShell/BSF style pane where the user can script the
> > > >   behaviour of WebDriver.
> > > >
> > > > I've got a very basic implementation working here:
> > > > https://github.com/cplim/JMeter - you'll have to switch to the
> > > 'webmeter'
> > > > branch to see the changes.  There are limitations with this proof of
> > > > concept, mainly:
> > > >
> > > >   1. It only creates Firefox instances
> > > >   2. I've only provided Javascript lang support using BSF.
> > > >
> > > > Some outstanding questions:
> > > >
> > > >   1. How to support Phone (Android/iOS) based WebDriver, as these
> seem
> > to
> > > >   expect to only connect to a single phone.
> > > >
> > > > I would be happy to contribute the changes back to JMeter.  Can I get
> > > some
> > > > feedback on this proposal, and what I need to do to contribute my
> > changes
> > > > back to the main JMeter codebase?
> > > >
> > > > regards,
> > > > CP Lim
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message