jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antonio Gomes Rodrigues <ra0...@gmail.com>
Subject Re: Graphite Listener
Date Wed, 01 Jan 2014 13:16:17 GMT
Hi,

In my opinion, graphite listener must be include in JMeter core for a lot
of reasons

1. All serious competitors (LoadRunner, neoload etc.) have it

2. It's avoid OutOfMemoryError by saving result in graphite and not in
memory

3. It's allow to have result in real time during very long test and not to
wait the end without tips

4. it's allow to have a complete monitoring with jmxtrans, collectd ...
(see my french article in
http://blog.aliecom.com/supervision-facile-avec-jmxtrans-collectd-et-graphite/)

5. It's allow to have result in real time during failure test (e.g. stop
the database, stop the cache during a load test)

6. It's allow to save the results to use it later (compare 2 run, ....)

etc.


2014/1/1 Philippe Mouawad <philippe.mouawad@gmail.com>

> On Tuesday, December 31, 2013, sebb wrote:
>
> > On 31 December 2013 17:55, Philippe Mouawad <philippe.mouawad@gmail.com
> <javascript:;>>
> > wrote:
> > > On Tue, Dec 31, 2013 at 5:24 PM, Milamber <milamber@apache.org
> <javascript:;>>
> > wrote:
> > >
> > >>
> > >> Le 31/12/2013 16:48, sebb a ecrit :
> > >>
> > >>  I'm not sure this belongs in JMeter at all.
> > >>>
> > >>> Just because it is useful to some, does not mean that it should be
> > >>> included.
> > >>>
> > >>
> > > One of the weaknesses of JMeter is the lack of graphs and historization
> > of
> > > these Graphs, Graphite listener is an answer to it.
> > >
> >
> > JMeter is not primarily intended as a reporting tool.
>
>
> Loot at all competitors they all have reporting and imho it is a mistake to
> ignore it.


I am ok with it


> Take the selection process of a load testing tool, what factor would you
> give to reports ?
> I would personally give a high one.
>

Me too

>
>
>
>
> > So long as there is sufficient reporting to be able to check that a
> > test is running OK, I think that is sufficient.
> >
>
> I disagree as currently summariser does not allow filtering on some
> samplers, for example take an ecommerce website, you might have a total of
> 1% error while on detail you have order pipeline with 30% of error but due
> to the low number of threads running it you won't see it until test has
> ended.
>

I am ok with it
The actual reporting is not enough


>
>
> > Proper analysis should take place offline after the test has completed.
>
> I agree but this feature allows also this.
> What about historisation this feature brings ?
>

Sometime we need feedback before the end of the test

And graphite allow it (real time reporting) and allow to save the results
to analyze it after the test has completed.

>
>
> >
> > What is perhaps missing currently is a way to save the output to a
> > database.
>
> But it's not enough yoy would still have to build reporting
>

I am ok with it.

Graphite allow to save and report the results easily.
If we have a way to save the output to a database, we need more works to
have a great report.

And this solution (to save the output to a database) don't allow to have
real time reporting easily



>
>
> >
> > >>> If we are not careful, JMeter is going to become a huge download
> (it's
> > >>> already quite big) containing lots of stuff that is only used by a
> few
> > >>> people.
> > >>>
> > >>
> > > I don't see currently useless elements , only maybe in Listeners:
> > > - Distribution Graph (alpha)
> > > - Spline Visualizer : consumes a lot of resources so not really usable
> > > - Graph Result : same argument
> > >
> > > What features do you think are unused, maybe we could do some cleanup.
> >
> > In retrospect, I think the BeanShell elements were probably unnecessary.
> > The same functionality is available using BSF and now JSR223.
> >
> > I've never used MongoDB so I don't know if that really belongs either.
> >
> > > Regarding my company, as big users of JMeter we nearly use all elements
> > > within JMeter either to script or load test except for LDAP, Mail and
> > > Listeners I mentionned.
> >
> > I've not used LDAP either, but it is a standard server interface.
> > Likewise Mail.
> >
> > However unfortunately once we have released additional code, it is not
> > going to be easy to drop it, as that might break test plans.
> > JMeter strives for upward compatibility.
> > This is another reason why it is important not to add extensions that
> > are not likely to be of long-term value.
> >
> > Yes, we have dropped some stuff that had become obsolete, but it is
> > not something to be undertaken lightly.
> >
> > > Also looking at various blogs I think nearly all elements are used.
> > >
> > > Regarding download Size, it will only increase by 1.2 mb and it is not
> an
> > > argument today taking into account the low price of disk space.
> >
> > It all adds up.
> >
> > >
> > >
> > >>> I am concerned that this is another addition that is specific some
> 3rd
> > >>> party software, rather than a generic solution which can be used with
> > >>> multiple 3rd party code.
> > >>>
> > >>
> > > Try to find generic solution for this, I don't see any.
> > >
> > >>
> > >>> The code is harder to test (because it needs a 3rd party installion)
> > >>> and therefore harder to maintain.
> > >>> It is dependent on the 3rd party API.
> > >>>
> > >> I don't think it is, look at code it is pretty simple in terms of
> > > features, it sends simple data using a simple format.
> > >
> > > Regarding Graphite Listener there is no real third party library , it
> is
> > > just a writing on a socket in a very simple format and it could be unit
> > > tested.
> > > commons-math3 is used to compute Percentile (it would be useful in a
> lot
> > of
> > > places) and commons-pool2 is to enable pooling of socket, it could in
> the
> > > future
> > > replace excalibur deprecated libraries
> > >
> > >>
> > >>> I think this is the sort of plugin that should probably be hosted
> > >>> externally.
> > >>> Not because of license issues, but because of the maintenance and
> size
> > >>> issues.
> > >>>
> > >>
> > >> Perhaps, the Apache Extras can be a good candidate to hosting this
> kind
> > of
> > >> plugins:
> > >> http://code.google.com/a/apache-extras.org/hosting/
> > >>
> > >> http://community.apache.org/apache-extras/faq.html
> > >>
> > >
> > > It would take us a lot of effort to start a project in it. And you
> can't
> > > ask user to go to X number of websites to have a full featured
> solution.
> > >
> > > I think we should open a poll on user mailing list to ask user their
> > > opinion and see what it gives.
> > >
> > > If we get no reaction, then I will propose it to a third party library
> > like
> > > jmeter-plugins as I think it is nearly always a companion of core
> JMeter
> > > due to the nice graphs at least.
> > >
> > >
> > >>
> > >> Milamber
> > >>
> > >>
> > >>
> > >>
> > >>> On 27 December 2013 16:25, Philippe Mouawad <
> > philippe.mouawad@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> ok, sorry for being impatient :)
> > >>>>
> > >>>> On Friday, December 27, 2013, Milamber wrote:
> > >>>>
> > >>>>  Le 27/12/2013 17:05, Philippe Mouawad a ecrit :
> > >>>>>
> > >>>>>  Hello,
> > >>>>>> Any feedback ?
> > >>>>>> Agree for commit ?
> > >>>>>>
> > >>>>>>  Please wait some days. The current period (christmas,
new year)
> > means
> > >>>>> often some vacation.
> > >>>>>
> > >>>>> And Graphite isn't easy to install/configure to test after
the
> > listener.
> > >>>>>
> > >>>>> (Some characters seems invalid in the french messages properties)
> > >>>>>
> > >>>>>
> > >>>>>  Regards
> > >>>>>>
> > >>>>>> On Thursday, December 26, 2013, Philippe Mouawad wrote:
> > >>>>>>
> > >>>>>>   Hello,
> > >>>>>>
> > >>>>>>> First Merry Christmas to you !
> > >>>>>>>
> > >>>>>>> Then a little note to notify that I created a bugzilla
for this
> > >>>>>>> feature
> > >>>>>>> and proposed a patch for it:
> > >>>>>>> - https://issues.apache.org/bugzilla/show_bug.cgi?id=55932
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Waiting for your remarks, reviewes and agreement to
commit it to
> > >>>>>>> JMeter
> > >>>>>>> trunk.
> > >>>>>>>
> > >>>>>>> We could mark it as beta or alpha for next release
if it seems to
> > you
> > >>>>>>> risky although it is totally isolated from the rest.
> > >>>>>>>
> > >>>>>>> Regards
> > >>>>>>> Philippe
> > >>>>>>>
> > >>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>


Antonio

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message