jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milamber <milam...@apache.org>
Subject Re: JMeter Performance evolution : My 2 cents
Date Fri, 25 May 2012 09:31:34 GMT
Hello,

Actuallly, the page http://wiki.apache.org/jmeter/JMeterPerformance returns
a http 500 error (others pages works)

Milamber

===
Internal Server Error

The server encountered an internal error or misconfiguration and was unable
to complete your request.

Please contact the server administrator at infrastructure@apache.org to
inform them of the time this error occurred, and the actions you performed
just before this error.

More information about this error may be available in the server error log.
------------------------------
Apache/2.4.1 (Unix) OpenSSL/1.0.0g Server at wiki.apache.org Port 80

===



On Fri, May 25, 2012 at 1:27 AM, sebb <sebbaz@gmail.com> wrote:

> On 24 May 2012 22:15, Philippe Mouawad <philippe.mouawad@gmail.com> wrote:
> > Hello,
> > How can I insert image ? button is disabled in GUI mode.
> > To insert Test Plan, is it an attachment ?
>
> Try clicking "Attachments" before starting the edit.
>
> > Thanks
> > Regards
> > Philippe
> >
> > On Wed, May 23, 2012 at 12:22 AM, sebb <sebbaz@gmail.com> wrote:
> >
> >> On 22 May 2012 20:18, Philippe Mouawad <philippe.mouawad@gmail.com>
> wrote:
> >> > Hello Milamber,
> >> > My responses below.
> >> > Regards
> >> > Philippe
> >> >
> >> > On Mon, May 21, 2012 at 11:51 PM, Milamber <milamber@apache.org>
> wrote:
> >> >
> >> >>
> >> >>
> >> >> Le 21/05/2012 22:12, Philippe Mouawad a ecrit :
> >> >> > Hello,
> >> >> > I read recently this little comparison :
> >> >> > https://github.com/excilys/gatling/wiki/Benchmarks<
> >> >> https://github.com/excilys/gatling/wiki/Benchmark>
> >> >> >
> >> >> > I reviewed the test plan that was used to make the test.
> >> >> > It seems to me the test is little biased:
> >> >> >
> >> >> >    - View Results Tree is in test plan (as it uses a lot of memory,
> >> it's
> >> >> a
> >> >> >    big issue)
> >> >> >    - View Results in Table (same thing)
> >> >> >    - 3 Non Standard JMeter listener, so It's not pure JMeter:
> >> >> >       - jp@gc - Transactions per Second
> >> >> >       - jp@gc - Response Times Over Time
> >> >> >       - jp@gc - Active Threads Over Time
> >> >> >    - Default XML output seems to have been used , it's against
best
> >> >> >    practices
> >> >> >
> >> >> >
> >> >> > Besides the following conclusions that seem to me non scientific
> and
> >> >> purely
> >> >> > subjective:
> >> >> >
> >> >> >    - The testers, new to both Gatling and JMeter found that JMeter
> was
> >> >> >    harder to learn and use than Gatling to create the simulations,
> >> >> despite the
> >> >> >    use of a proxy.
> >> >> >
> >> >> >
> >> >> > I will only look at other ideas expressed:
> >> >> >
> >> >> >    - JMeter creates one thread per user simulated. If there is
not
> >> enough
> >> >> >    memory allocated to the JVM, it can crash trying to create
these
> >> >> threads
> >> >> >       - This need to be detailed, cause either it fails with OOM
> and
> >> it's
> >> >> >       not during thread creation, either it fails with "unable
to
> >> create
> >> >> new
> >> >> >       native thread"
> >> >> >    - For instance, JMeter could not run 1500 users with 512 MB
> (what
> >> was
> >> >> >    used for Gatling even with 2000 users); OutOfMemoryErrors are
> >> >> recorded in
> >> >> >    the table as *OOM*
> >> >> >       - => I made a Test with up to 2000 Threads with 512 m
without
> >> any
> >> >> >       crash, it depends on Test and on application
> >> >> >    - Another problem occurred with the 2000 users simulations;
it
> >> seems
> >> >> >    that JMeter can not simulate more than 1514 users independently
> >> from
> >> >> the
> >> >> >    memory that was allocated to the JVM
> >> >> >       - => I made a Test with up to 2000 Threads with 512 m
without
> >> any
> >> >> >       crash, so assertion is false, it depends on Test and on
> >> application
> >> >> >
> >> >> >
> >> >> > As it's difficult to install the application used for test (last
> >> version
> >> >> > does not seem to work as expected) , if they provide a working
WAR
> >> >> against
> >> >> > a local Postgres DB I will be happy to test with it.
> >> >> > But in current state, application seems to be packaged for cloud
> or H2
> >> >> > local DB, I didn't want to spend too much time setting up
> application
> >> as
> >> >> I
> >> >> > don't know its real status.
> >> >> >
> >> >> > I just tried to run Test Plan against a blank tomcat to verify
what
> >> they
> >> >> > say about Thread Creation, I didn't find any issue on this.
> >> >> >
> >> >> > So I decided to make a very simple scenario test on Tomcat Examples
> >> (It
> >> >> > goes to Session Example, adds attribute, go back to index, go
back
> to
> >> >> > Session Example, test contains Response assertion for each
> Request).
> >> >> >
> >> >>
> >> >> Please, indicate the Tomcat version used? it's the same machine?
> config
> >> >> of tomcat server? tuning of JVM for Tomcat? OS? OS tuning? JVM
> editor?
> >> >>
> >> >> Tomcat apache-tomcat-6.0.24
> >> > -Xms256m -Xmx1024m
> >> >    <Connector port="8080" protocol="HTTP/1.1"
> >> >
> >> >
> >>
> compressableMimeType="text/html,text/xml,text/plain,text/javascript,application/json"
> >> >                compression="off"
> >> >                socketBuffer="8"
> >> >                maxThreads="400"
> >> >               connectionTimeout="20000"
> >> >               redirectPort="8443" />
> >> > Set session timeout in web.xml  to 1min
> >> > java version "1.6.0_29"
> >> > Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527)
> >> > Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> >> >
> >> > Mac OS 10.6.8
> >> > JMeter  and Tomcat on same machine
> >> > No particular OS Tuning
> >> > 8GO RAM
> >> > No swap, nothing runnning except these 2
> >> > Tomcat CPU around 5%
> >> > Memory around 50 mo
> >> >
> >> > JVM editor/version for your JMeter? OS? OS Tuning (TCP tuning?)?
> >> >>
> >> >> Same config
> >> > No TCP tuning
> >> >
> >> >> Can you post the jmx file?
> >> >>
> >> >> Will it be accepted on this list ?
> >>
> >> Best to create a Wiki page and attach the test file there.
> >>
> >> >
> >> >> Milamber
> >> >>
> >> >>
> >> >> > It is not at all representative but it is a way for me to check
> >> potential
> >> >> > issues in JMeter and performance changes accross versions.
> >> >> >
> >> >> > I ran the test with 1500 VU using JMeter 2.5.1,  2.7 (current
> trunk)
> >> with
> >> >> > -Xmx512m, 10 minutes run and CSV output against a local Tomcat
(I
> >> restard
> >> >> > tomcat between tests and control its health):
> >> >> >
> >> >> >    - I noticed that current trunk version behaves much better
in
> >> terms of
> >> >> >    memory than 2.5.1 or 2.6:
> >> >> >       - In 2.5.1 :
> >> >> >       - GC activity is much higher with around 5 GC CPU peaks
> every 2
> >> >> >          minutes,  and 20 FULL GC of 700 to 800 ms each
> >> >> >          - Throughput: 97,71%
> >> >> >          - Pauses : 13,69s
> >> >> >          - Mém : 391M/min
> >> >> >          - Full GC tend to be much more frequent at end of test
> >> >> >          - 2.7:
> >> >> >          -  no GC CPU peak, 1 FULL GC
> >> >> >          - Throughput:98.54
> >> >> >          - Pauses : 8.9s
> >> >> >          - 1108m /min
> >> >> >          - Summary:
> >> >> >       - 25.1:
> >> >> >          - 164676 samples in 605,1s
> >> >> >          - 272,2/s
> >> >> >          - Avg:    97
> >> >> >          - 2.7:
> >> >> >          - 165367 in 605.0s
> >> >> >          - 273.3/s
> >> >> >          - Avg:   228
> >> >> >          - I also noticed results have a much better look:
> >> >> >       - in 2.5.1, Transactions/s are around 300 / sec during 4
> >> minutes,
> >> >> >       then drop to 200/s, go up to 400/s , then down to 260/s
and
> >> >> > finally 200/sec
> >> >> >       - in 2.7, Transactions stay around 300/sec
> >> >> >
> >> >> >
> >> >> >
> >> >> > Actions:
> >> >> >
> >> >> >    - I think it would be useful to have some reference application
> on
> >> >> which
> >> >> >    we could test JMeter behaviour
> >> >> >    - What would the best place to put these comparisons ? wiki
?
> which
> >> >> >    indicators should we put ?
> >> >> >    - Further testing should be done against a richer application
> that
> >> >> could
> >> >> >    be deployed locally
> >> >> >
> >> >> >
> >> >> > Regards
> >> >> > Philippe
> >> >> >
> >> >> >
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > Cordialement.
> >> > Philippe Mouawad.
> >>
> >
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message