commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Artem Barger <>
Subject Re: [rng] Usefulness of benchmarks
Date Sat, 03 Sep 2016 21:33:32 GMT
On Sat, Sep 3, 2016 at 1:36 AM, Gilles <> wrote:

> The discrepancy between "PerfTestUtils" and JMH could be a bug (in
> "PerfTestUtils" of course!) or ... measuring different use-cases:
> Use of several RNGs at the same time vs using a single one; the
> latter could allow for more aggressive optimizations.

​I'm not really familiar with the PerfTestUtils, while I know that JMH is
doing a great
job to avoid different pitfalls while building microbenchmarks for
measuring a performance.
Also it looks a bit suspicious where comparing JDK random generator against
itself it's not
showing ration of 1.0 for PerfTestUtils.

> Lacking input as to what the benchmarks purport to demonstrate, I'm
> about to simply delete the "PerfTestUtils" column.
> The result will be a simplified (maybe simplistic) view of the
> relative performance of the RNGs in the "single use" use-case.
​I can try to take a look on PerfTestUtils to understand what is the main
cause of such difference.​

> Any comment, objection, explanation, suggestion?
> [E.g. set up JMH to benchmark the other use case, or a reason why
> this is in fact not necessary.]

​We can play with different amount of warm up rounds in JMH to see whenever
there is a degradation
to results similar to PerfTestUtils for example.​

Best regards,
                      Artem Barger.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message