jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: New Listener to generate HTML report at end of load test Was APDEX Computing and reporting
Date Tue, 03 Feb 2015 13:06:56 GMT
On 3 February 2015 at 12:38, Philippe Mouawad
<philippe.mouawad@gmail.com> wrote:
> Hi,
> Any additional opinions of dev team or users ?
> I 'd like to know if I should continue efforts on this or stop it.

I am not saying it should not be included with JMeter.

I just don't think it belongs in the main application, for the reasons
already given.

> Thanks
> Regards
>
> On Monday, February 2, 2015, Philippe Mouawad <philippe.mouawad@gmail.com>
> wrote:
>
>>
>>
>> On Monday, February 2, 2015, sebb <sebbaz@gmail.com
>> <javascript:_e(%7B%7D,'cvml','sebbaz@gmail.com');>> wrote:
>>
>>> On 1 February 2015 at 22:41, Philippe Mouawad
>>> <philippe.mouawad@gmail.com> wrote:
>>> > On Sunday, February 1, 2015, sebb <sebbaz@gmail.com> wrote:
>>> >
>>> >> On 1 February 2015 at 19:31, Philippe Mouawad
>>> >> <philippe.mouawad@gmail.com <javascript:;>> wrote:
>>> >> > Hi ,
>>> >> > I uploaded a screenshot and new patch code showing what I meant.
>>> >> > I think including it in core should be considered seriously.
>>> >>
>>> >> If the code reads the CSV file at the end of a test to produce the
>>> >> HTML file, there is no need for it to be included in the JMeter code
>>> >> which is used to run tests.
>>> >> It should be a separate application.
>>> >>
>>> >> I don't share your opinion sebb.
>>> > Having a report ready at end of load test is nowadays a standard, look
>>> at
>>> > market tools wether open source or not.
>>> > I don't see the problem here, can you give more details ?
>>>
>>> The problem is that it adds extra code to the JMeter application which
>>> takes extra memory.
>>
>>
>> My last implementation only uses memory at end of test not morz during it.
>>
>>
>>> Also, if the user wants to process existing files, it will be easier
>>
>> to have a standalone application rather than having to start the
>>> JMeter test application and then find the reporting tool.
>>
>>
>> It is possible with it, just make a plan with 1 debug sampler and put
>> listener. Report will be generated.
>>
>>>
>>> It would be useful if the tool could be used in batch mode to process
>>> a set of CSV files.
>>
>>
>> Maybe, but I think it's an additional feature. Also having another tool
>> may lead to what happened to the report package, it seems it was never
>> used.  But if you implement it it will be ok for me.
>>
>>
>>>
>>> It's also likely to be easier for others to contribute additional
>>> reporting tools if they are part of a separate application.
>>
>> I don't understand this ? how is it easier if it's in jmeter core. If it's
>> apart I don't support this option.
>>
>>
>>
>>> It should make the coding requirements simpler.
>>
>> how ?
>>
>>>
>>> > Having another application( by the way do you mean provided as a tool in
>>> > jmeter or third party) means in the first case you need to setup another
>>> > tool and in the latter case need to develop your own, or use or pay one.
>>>
>>> Of course the code should be included with JMeter.
>>> But it should be a separate tool.
>>>
>>> > This is the major drawback of JMeter I hear from customers and read
>>> around
>>> > the net.
>>>
>>> > I first though it would not be that easy but it appears we are able to
>>> > reuse existing code.
>>>
>>> I don't follow.
>>>
>>> > If you disagree, I think we should wait for other team members their
>>> > opinion and/or setup a vote for this.
>>>
>>> The main JMeter code should be reserved for setting up and running tests.
>>>
>> IMHO reporting must be part of JMeter, and anyway it is already through
>> some Listeners , which although not perfect bring useful infos. The aim of
>> this new listener is:
>> 1/ Allow generation in gui and non gui mode
>> 2/ Have a "sexy" (not yet ) report readable in browser with dynamic
>> behaviour (zoom, select some samples...)
>> 3/ Allow generation at end of load test
>> 4/ Make it easily customizable as FTL will be a property
>> 5/ Make it extensible in the future
>>
>>
>>
>>
>>> >
>>> >
>>> >
>>> >
>>> >> > @Felix thanks for review of initial patch.
>>> >> > Regards
>>> >> >
>>> >> > On Sunday, February 1, 2015, sebb <sebbaz@gmail.com <javascript:;>>
>>> >> wrote:
>>> >> >
>>> >> >> On 31 January 2015 at 13:16, Philippe Mouawad
>>> >> >> <philippe.mouawad@gmail.com <javascript:;> <javascript:;>>
wrote:
>>> >> >> > On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher <
>>> >> >> > felix.schumacher@internetallee.de <javascript:;>
<javascript:;>>
>>> >> wrote:
>>> >> >> >
>>> >> >> >> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad:
>>> >> >> >>
>>> >> >> >>> Hi,
>>> >> >> >>> I intend to commit a BackendListener client implementation
that
>>> >> >> computes
>>> >> >> >>> APDEX at end of Load Test.
>>> >> >> >>> It will take:
>>> >> >> >>>
>>> >> >> >>>     - Acceptable Response Time (http://www.apdex.fr/)
T
>>> >> >> >>>
>>> >> >> >> The link was in french (my french is not very good
(really non
>>> >> existant
>>> >> >> >> :)), but luckily wikipedia had an article about apdex,
which I
>>> could
>>> >> >> read.
>>> >> >> >>
>>> >> >> >
>>> >> >> > Sorry for the french link some references:
>>> >> >> > http://www.apdex.org/overview.html
>>> >> >> >
>>> >> >> >      - Compute F as 4xT but allow customization
>>> >> >> >>>     - List of samples taken into account
>>> >> >> >>>
>>> >> >> >>>
>>> >> >> >>> This listener will compute it during load test
and generate an
>>> HTML
>>> >> >> file
>>> >> >> >>> at
>>> >> >> >>> end of test containing it.
>>> >> >> >>>
>>> >> >> >>> This listener could be later enhanced to :
>>> >> >> >>>
>>> >> >> >>>     - Generate equivalent of Aggregate Report
>>> >> >> >>>     - Possibly graphs based on chartjs or some
other graphing JS
>>> >> >> Library
>>> >> >> >>>
>>> >> >> >> I understand, that apdex is a simple mean to have
one metric. As
>>> >> such I
>>> >> >> >> wonder why we would want to graph it.
>>> >> >> >
>>> >> >> > In fact my proposal was larger than just APDEX.
>>> >> >> > What I am proposing is to implement a report generator
that at
>>> end of
>>> >> >> Test
>>> >> >> > would compute HTML page with:
>>> >> >> > - APDEX
>>> >> >> > - Some graphs like:
>>> >> >> > - Plot of OK/KO  (http://www.chartjs.org/docs/#doughnut-pie-chart
>>> )
>>> >> >> > - Request Summary (what's currently in Aggregate Graph)
>>> >> >> > http://www.chartjs.org/docs/#bar-chart
>>> >> >> > - Response Time over time (
>>> http://www.chartjs.org/docs/#line-chart)
>>> >> >> >
>>> >> >> > I think all this should in fact be computed at end of
load test
>>> >> instead
>>> >> >> of
>>> >> >> > during it.
>>> >> >> > The input of this listener would be the generated CSV
file, and at
>>> >> end of
>>> >> >> > Test it would read it an compute all this.
>>> >> >>
>>> >> >> In which case it should be a stand-alone app.
>>> >> >>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >> If we want to display a graph of apdex values for
different
>>> tolerated
>>> >> >> >> times, than we would have to store all values, which
would make
>>> the
>>> >> >> >> listener quite heavy on the memory side.
>>> >> >> >>
>>> >> >> >>>
>>> >> >> >>> Thoughts ?
>>> >> >> >>>
>>> >> >> >> How will errors be counted? Sometimes I can cope with
errors, as
>>> >> long as
>>> >> >> >> they are reported fast and sometimes an error would
result in a
>>> very
>>> >> >> >> unhappy  consumer.
>>> >> >> >>
>>> >> >> > Good catch, I think only successful samples must be taken
into
>>> >> account.
>>> >> >> > Error will count only toward total requests (unsatisfied).
>>> >> >> >
>>> >> >> >>
>>> >> >> >> Regards
>>> >> >> >>  Felix
>>> >> >> >>
>>> >> >> >>
>>> >> >> >
>>> >> >> >
>>> >> >> > --
>>> >> >> > Cordialement.
>>> >> >> > Philippe Mouawad.
>>> >> >>
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Cordialement.
>>> >> > Philippe Mouawad.
>>> >>
>>> >
>>> >
>>> > --
>>> > Cordialement.
>>> > Philippe Mouawad.
>>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>>
>
> --
> Cordialement.
> Philippe Mouawad.

Mime
View raw message