jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Schumacher <felix.schumac...@internetallee.de>
Subject Re: [GitHub] jmeter issue #296: Bug 61078 - Percentile calculation error
Date Sat, 03 Jun 2017 13:59:59 GMT
Am 03.06.2017 um 10:55 schrieb Philippe Mouawad:
> On Wed, May 31, 2017 at 2:54 PM, Vladimir Sitnikov <
> sitnikov.vladimir@gmail.com> wrote:
>
>> Philippe>- switch everywhere to R1 (also in commons-math)
>>
>> Can you please clarify why do you prefer R1?
>>
> Because from what the reporter wrote, it looked good to me:
> "If the 90% percentile is 1200 ms than that means that 90% of tests take no
> more than 1200 ms"
>
> And there is a another pragmatic point, it seems JOrphan implementation is
> R1 once we fix the issue.
>
>
>
>> I'm inclined to R8 (as it is recommended by R for sample quantile
>> calculation).
>>
>> 1) I think interpolation would reduce run-to-run variance.
>> 2) Interpolation-like estimation is easier to implement. For instance, if
>> HdrHistogram estimator is added, then its result would be closer to R8
>> rather than to R1.
>>
> ok
>
>
>> I don't think "the result of 90% is one of the sample response times" is
>> important. The important stuff is how system under test behaves, and it is
>> not something tied to a single execution. What I mean is R8 kind of
>> computation should better approximate the true percentile value than R1
>> would, and it is the true value that is important to compare and report as
>> a test result.
>>
> Will you submit a PR for that ? Before or after release of 3.3 ?
Is there any need to rush this before 3.3?

I still believe, that there is no exact definition for the median, but 
there are may specialised definitions like the R1 .. R8 mentioned by 
Vladimir. So I think, that whatever we chose now, will be challenged by 
someone in the future.

If we change the algorithm, we should document it well and try to make 
it so flexible, that it can be configured to act as R1 .. R8 (if that is 
possible without too much work).

If we don't change the algorithm, we should document the current state.

Felix

> Thanks
>
>
>> Vladimir
>>
>
>


Mime
View raw message