jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shmuel Krakower <shmul...@gmail.com>
Subject Re: Web Driver based Sampler?
Date Sun, 22 Apr 2012 13:49:23 GMT
I'd suggest to HP to drop C language scripting in order to make scripting
easier / shorter.

As Chaitanya said - browser response times is overrated.

Anyhow, I agree that implementing load tests with hundreds / thousands of
virtual users with that kind of a sampler(which running a browser) will
require much more hardware but as CP said, with today's cloud services this
is very affordable, thus this is not the issue.
I think that there are services on the web which already doing this (
https://browsermob.com/website-load-testing-features), but I never tried
those.

CP, if you get different numbers between you monitoring solution and JMeter
reports, you should try to understand why that happens.
It might be for many reasons. Did developing this sampler resolved your
original problem? Do you get more close numbers now?

Shmuel.


On Sun, Apr 22, 2012 at 5:27 AM, Cheen-Pin Lim <cheenpin.lim@gmail.com>wrote:

> Hi Chaitanya,
>
> I guess I am looking at this from a browser view rather than from the
> server side.  What I tend to find is that current load testing does not
> properly reflect the browser load times on our site.  The reports from
> Google (in webmasters tools) and newrelic.com give a very different
> picture
> from that done during load/performance testing.
>
> We use HP Load Runner for our in-house load testing and the numbers we get
> (even when we download 'all' external resources) does not come close to the
> numbers reported by the other tools.  I guess I am attempting to address a
> gap that I see in our in-house load/performance testing, and the
> integration of webdriver into JMeter could solve this problem.  At this
> stage it is an attempt to try to provide a means to simulate what our
> production servers see (i.e. using a real browser that will spend time
> executing JS/CSS).
>
> Yes, I agree that the complexity and cost of running a browser instance is
> higher than using the standard HTTP Sampler, but with modern services such
> as Amazon AWS, I would think that these problems would be
> addressable/solvable.
>
> regards,
> CP
>
> On 22 April 2012 06:30, chaitanya bhatt <bhatt.chaitanya@gmail.com> wrote:
>
> > *On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim <cheenpin.lim@gmail.com
> > >wrote:
> > >Once you have more load
> > >applied to the servers/services different characteristics tend to
> emerge,
> > >which is why I was thinking that the use of JMeter would be useful.
> >
> > *I may totally sound cynical while saying this but honestly the reason
> why
> > I am criticising your idea is mainly due to the week rational behind your
> > motivation to create this feature:
> >
> > 1. First of all browser induced latency on the total workload is grossly
> > overrated. Having said that, all that an accurate simulation of browser
> > latency would do to your workload is that it would marginally reduce the
> > total hits made on the server, which I think is bad from a fundamental
> load
> > testing perspective.
> > 2.To accommodate in-memory containers to hold 100s-1000s of virutal user
> > browser sessions is practically impossible without testers having to
> > significantly size up their test lab resources. Think about the cost
> > factor.
> > 3. Looking at todays trend of browser adoption your web driver would have
> > to be equipped with simulation capability multiple types of browsers such
> > as IE, Chromer, Safari etc. etc. and also you would have to consider
> > simulating various device characteristics on the generated load. Think
> > about the complexity.
> >
> > @Shmuel: The loadrunner protocol you are talking about is Ajax
> > TrueClient. HP came out with Ajax TrueClient not with an intention
> > of generating real browser traffic but it was mainly developed to reduce
> > the complexity of scripting apps that employ unintelligible data
> > serialization like Google GWT. With this HP's protocol you wouldn't have
> a
> > necessity of correlating session variables at all and there by you would
> be
> > significantly reducing the time to develop scripts. However, this
> protocol
> > currently has poor user adoption due to various limitations.
> >
> > Thanks
> > Chaitanya M Bhatt
> > http://www.performancecompetence.com
> >
> > On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim <cheenpin.lim@gmail.com
> > >wrote:
> >
> > > Hi,
> > >
> > > The tools mentioned help a great deal, however they generally look at
> it
> > > from a single browser to a single server scenario.  Once you have more
> > load
> > > applied to the servers/services different characteristics tend to
> emerge,
> > > which is why I was thinking that the use of JMeter would be useful.
> > >
> > > Furthermore, if our primary audience is hitting the site using a
> browser,
> > > then would it not make sense to use the same tool to simulate the load?
> > >
> > > Just my thoughts.
> > >
> > > regards,
> > > CP
> > >
> > > On 21 April 2012 17:26, chaitanya bhatt <bhatt.chaitanya@gmail.com>
> > wrote:
> > >
> > > > Pleanty of awesome tools are already available for free to meet your
> > > goals.
> > > >
> > > > Free:
> > > > https://developers.google.com/pagespeed/
> > > > http://yslow.org/
> > > >
> > > > Free or Premium:
> > > > http://ajax.dynatrace.com/ajax/en/content/c-speed-page-load.aspx
> > > >
> > > >
> > > > Selenium has a webdriver which you can plan on extending - if you
> will.
> > > > IMHO Jmeter is not the right platform for this.
> > > > Thanks
> > > > Chaitnaya M Bhatt
> > > > http://www.performancecompetence.com
> > > > On Fri, Apr 20, 2012 at 11:55 PM, Cheen-Pin Lim <
> > cheenpin.lim@gmail.com
> > > > >wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > After looking at the current HTTP Sampler, I would like to propose
> a
> > > > > WebDriver based implementation.  The problem is that with a lot of
> > > modern
> > > > > websites using AJAX, advanced CSS3 (transitions/transforms) to do
> the
> > > > > rendering, the performance problem may not be 'visible' to the
> > standard
> > > > > HTTP Sampler.  The idea is that if we could use JMeter to control
a
> > > > browser
> > > > > and collect the time it takes to render each 'page' it would give
> the
> > > > users
> > > > > an understanding of how long a page would take to render.
> > > > >
> > > > > Some of the high level goals are as follows:
> > > > >
> > > > >   1. Allow any WebDriver supported browser to be used to perform
> the
> > > > >   Sampling.
> > > > >   2. Provide a BeanShell/BSF style pane where the user can script
> the
> > > > >   behaviour of WebDriver.
> > > > >
> > > > > I've got a very basic implementation working here:
> > > > > https://github.com/cplim/JMeter - you'll have to switch to the
> > > > 'webmeter'
> > > > > branch to see the changes.  There are limitations with this proof
> of
> > > > > concept, mainly:
> > > > >
> > > > >   1. It only creates Firefox instances
> > > > >   2. I've only provided Javascript lang support using BSF.
> > > > >
> > > > > Some outstanding questions:
> > > > >
> > > > >   1. How to support Phone (Android/iOS) based WebDriver, as these
> > seem
> > > to
> > > > >   expect to only connect to a single phone.
> > > > >
> > > > > I would be happy to contribute the changes back to JMeter.  Can I
> get
> > > > some
> > > > > feedback on this proposal, and what I need to do to contribute my
> > > changes
> > > > > back to the main JMeter codebase?
> > > > >
> > > > > regards,
> > > > > CP Lim
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message