jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe Mouawad <philippe.moua...@gmail.com>
Subject New Listener to generate HTML report at end of load test Was APDEX Computing and reporting
Date Tue, 03 Feb 2015 12:38:05 GMT
Hi,
Any additional opinions of dev team or users ?
I 'd like to know if I should continue efforts on this or stop it.

Thanks
Regards

On Monday, February 2, 2015, Philippe Mouawad <philippe.mouawad@gmail.com>
wrote:

>
>
> On Monday, February 2, 2015, sebb <sebbaz@gmail.com
> <javascript:_e(%7B%7D,'cvml','sebbaz@gmail.com');>> wrote:
>
>> On 1 February 2015 at 22:41, Philippe Mouawad
>> <philippe.mouawad@gmail.com> wrote:
>> > On Sunday, February 1, 2015, sebb <sebbaz@gmail.com> wrote:
>> >
>> >> On 1 February 2015 at 19:31, Philippe Mouawad
>> >> <philippe.mouawad@gmail.com <javascript:;>> wrote:
>> >> > Hi ,
>> >> > I uploaded a screenshot and new patch code showing what I meant.
>> >> > I think including it in core should be considered seriously.
>> >>
>> >> If the code reads the CSV file at the end of a test to produce the
>> >> HTML file, there is no need for it to be included in the JMeter code
>> >> which is used to run tests.
>> >> It should be a separate application.
>> >>
>> >> I don't share your opinion sebb.
>> > Having a report ready at end of load test is nowadays a standard, look
>> at
>> > market tools wether open source or not.
>> > I don't see the problem here, can you give more details ?
>>
>> The problem is that it adds extra code to the JMeter application which
>> takes extra memory.
>
>
> My last implementation only uses memory at end of test not morz during it.
>
>
>> Also, if the user wants to process existing files, it will be easier
>
> to have a standalone application rather than having to start the
>> JMeter test application and then find the reporting tool.
>
>
> It is possible with it, just make a plan with 1 debug sampler and put
> listener. Report will be generated.
>
>>
>> It would be useful if the tool could be used in batch mode to process
>> a set of CSV files.
>
>
> Maybe, but I think it's an additional feature. Also having another tool
> may lead to what happened to the report package, it seems it was never
> used.  But if you implement it it will be ok for me.
>
>
>>
>> It's also likely to be easier for others to contribute additional
>> reporting tools if they are part of a separate application.
>
> I don't understand this ? how is it easier if it's in jmeter core. If it's
> apart I don't support this option.
>
>
>
>> It should make the coding requirements simpler.
>
> how ?
>
>>
>> > Having another application( by the way do you mean provided as a tool in
>> > jmeter or third party) means in the first case you need to setup another
>> > tool and in the latter case need to develop your own, or use or pay one.
>>
>> Of course the code should be included with JMeter.
>> But it should be a separate tool.
>>
>> > This is the major drawback of JMeter I hear from customers and read
>> around
>> > the net.
>>
>> > I first though it would not be that easy but it appears we are able to
>> > reuse existing code.
>>
>> I don't follow.
>>
>> > If you disagree, I think we should wait for other team members their
>> > opinion and/or setup a vote for this.
>>
>> The main JMeter code should be reserved for setting up and running tests.
>>
> IMHO reporting must be part of JMeter, and anyway it is already through
> some Listeners , which although not perfect bring useful infos. The aim of
> this new listener is:
> 1/ Allow generation in gui and non gui mode
> 2/ Have a "sexy" (not yet ) report readable in browser with dynamic
> behaviour (zoom, select some samples...)
> 3/ Allow generation at end of load test
> 4/ Make it easily customizable as FTL will be a property
> 5/ Make it extensible in the future
>
>
>
>
>> >
>> >
>> >
>> >
>> >> > @Felix thanks for review of initial patch.
>> >> > Regards
>> >> >
>> >> > On Sunday, February 1, 2015, sebb <sebbaz@gmail.com <javascript:;>>
>> >> wrote:
>> >> >
>> >> >> On 31 January 2015 at 13:16, Philippe Mouawad
>> >> >> <philippe.mouawad@gmail.com <javascript:;> <javascript:;>>
wrote:
>> >> >> > On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher <
>> >> >> > felix.schumacher@internetallee.de <javascript:;> <javascript:;>>
>> >> wrote:
>> >> >> >
>> >> >> >> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad:
>> >> >> >>
>> >> >> >>> Hi,
>> >> >> >>> I intend to commit a BackendListener client implementation
that
>> >> >> computes
>> >> >> >>> APDEX at end of Load Test.
>> >> >> >>> It will take:
>> >> >> >>>
>> >> >> >>>     - Acceptable Response Time (http://www.apdex.fr/)
T
>> >> >> >>>
>> >> >> >> The link was in french (my french is not very good (really
non
>> >> existant
>> >> >> >> :)), but luckily wikipedia had an article about apdex,
which I
>> could
>> >> >> read.
>> >> >> >>
>> >> >> >
>> >> >> > Sorry for the french link some references:
>> >> >> > http://www.apdex.org/overview.html
>> >> >> >
>> >> >> >      - Compute F as 4xT but allow customization
>> >> >> >>>     - List of samples taken into account
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> This listener will compute it during load test and
generate an
>> HTML
>> >> >> file
>> >> >> >>> at
>> >> >> >>> end of test containing it.
>> >> >> >>>
>> >> >> >>> This listener could be later enhanced to :
>> >> >> >>>
>> >> >> >>>     - Generate equivalent of Aggregate Report
>> >> >> >>>     - Possibly graphs based on chartjs or some other
graphing JS
>> >> >> Library
>> >> >> >>>
>> >> >> >> I understand, that apdex is a simple mean to have one
metric. As
>> >> such I
>> >> >> >> wonder why we would want to graph it.
>> >> >> >
>> >> >> > In fact my proposal was larger than just APDEX.
>> >> >> > What I am proposing is to implement a report generator that
at
>> end of
>> >> >> Test
>> >> >> > would compute HTML page with:
>> >> >> > - APDEX
>> >> >> > - Some graphs like:
>> >> >> > - Plot of OK/KO  (http://www.chartjs.org/docs/#doughnut-pie-chart
>> )
>> >> >> > - Request Summary (what's currently in Aggregate Graph)
>> >> >> > http://www.chartjs.org/docs/#bar-chart
>> >> >> > - Response Time over time (
>> http://www.chartjs.org/docs/#line-chart)
>> >> >> >
>> >> >> > I think all this should in fact be computed at end of load
test
>> >> instead
>> >> >> of
>> >> >> > during it.
>> >> >> > The input of this listener would be the generated CSV file,
and at
>> >> end of
>> >> >> > Test it would read it an compute all this.
>> >> >>
>> >> >> In which case it should be a stand-alone app.
>> >> >>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >> If we want to display a graph of apdex values for different
>> tolerated
>> >> >> >> times, than we would have to store all values, which would
make
>> the
>> >> >> >> listener quite heavy on the memory side.
>> >> >> >>
>> >> >> >>>
>> >> >> >>> Thoughts ?
>> >> >> >>>
>> >> >> >> How will errors be counted? Sometimes I can cope with
errors, as
>> >> long as
>> >> >> >> they are reported fast and sometimes an error would result
in a
>> very
>> >> >> >> unhappy  consumer.
>> >> >> >>
>> >> >> > Good catch, I think only successful samples must be taken
into
>> >> account.
>> >> >> > Error will count only toward total requests (unsatisfied).
>> >> >> >
>> >> >> >>
>> >> >> >> Regards
>> >> >> >>  Felix
>> >> >> >>
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Cordialement.
>> >> >> > Philippe Mouawad.
>> >> >>
>> >> >
>> >> >
>> >> > --
>> >> > Cordialement.
>> >> > Philippe Mouawad.
>> >>
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
>

-- 
Cordialement.
Philippe Mouawad.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message