commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gilles Sadowski <gillese...@gmail.com>
Subject Re: [rng] BigCrush results
Date Fri, 11 Oct 2019 14:45:14 GMT
Hello.

Le ven. 11 oct. 2019 à 15:54, Alex Herbert <alex.d.herbert@gmail.com> a écrit :
>
>
> On 07/10/2019 15:00, Gilles Sadowski wrote:
> > Hi.
> >
> > Le lun. 7 oct. 2019 à 15:54, Alex Herbert <alex.d.herbert@gmail.com> a écrit
:
> >> I have found a bug in the stress test application that affects all the
> >> BigCrush results.
> >>
> >> This was found when updating the stress test application to support
> >> 64-bit testing within PractRand, and testing of upper and lower 32-bits
> >> from 64-bit out. An initial attempt to show that the lower 32-bits, bit
> >> reversed, of the XOR_SHIFT_1024_S systematically fail Linear Complexity
> >> tests did not work.
> >>
> >> The bug was a dual reversal of the platform byte-order. Code was added
> >> to reverse the byte order in the link c application that read from stdin
> >> and passes numbers to BigCrush. Later code was added to reverse the byte
> >> order in the Java application so allowing Dieharder to work. The
> >> original c application should have been updated but was not (despite the
> >> fact I thought I had done this). So the byte order was not correct after
> >> a dual reversal and the BigCrush results refer to a byte reversed output
> >> sequence.
> >>
> >> I fixed the code and have verified that the stress test application can
> >> identify Linear Complexity failures in XOR_SHIFT_1024_S.
> >>
> >> I am rerunning BigCrush for the other generators.
> >>
> >> Testing of generators with PractRand is still under trial and will
> >> probably not make it for the 1.3 release. The only outstanding item for
> >> 1.3 is continued work on updating the user guide.
> > The userguide (and web site) can be updated at any time (between
> > releases); it is not be a blocker.
>
> BigCrush has finished. Not much changed.
>
> I've added an option to look for systematic failures to the stress test
> application. It is shown in the below table using the intersection
> symbol (view these tables using a fixed width font):
>
> RNG                      New                   ∩          Old                   ∩
> JDK                      50,51,52,49,51        48         74,72,74,73,74        70
> WELL_512_A               6,7,8,6,6             6          7,6,6,6,6             6
> WELL_1024_A              5,4,5,5,4             4          4,4,5,4,4             4
> WELL_19937_A             2,2,3,3,3             2          3,3,2,2,2             2
> WELL_19937_C             4,2,2,2,2             2          2,2,3,2,2             2
> WELL_44497_A             3,2,2,2,3             2          2,2,2,2,3             2
> WELL_44497_B             2,2,2,2,2             2          2,3,2,2,2             2
> MT                       2,3,2,2,3             2          2,3,2,2,2             2
> ISAAC                    1,1,0,0,1             0          0,1,2,0,0             0
> SPLIT_MIX_64             0,0,0,1,0             0          1,0,0,0,0             0
> XOR_SHIFT_1024_S         1,0,0,0,2             0          0,0,0,0,0             0
> TWO_CMRES                0,1,0,0,0             0          4,3,3,5,4             3
> MT_64                    2,2,2,2,2             2          2,3,2,2,2             2
> MWC_256                  0,1,1,1,0             0          0,0,0,2,0             0
> KISS                     1,1,0,0,0             0          0,0,0,0,0             0
> XOR_SHIFT_1024_S_PHI     0,2,0,0,1             0          0,0,0,0,0             0
> XO_RO_SHI_RO_64_S        1,2,3,1,1             1          1,1,2,1,3             1
> XO_RO_SHI_RO_64_SS       0,1,0,0,0             0          0,0,0,0,0             0
> XO_SHI_RO_128_PLUS       0,0,1,0,0             0          1,2,2,1,1             1
> XO_SHI_RO_128_SS         1,0,1,0,0             0          0,1,0,0,0             0
> XO_RO_SHI_RO_128_PLUS    1,0,0,0,0             0          0,1,0,0,0             0
> XO_RO_SHI_RO_128_SS      1,1,1,0,0             0          1,0,1,0,0             0
> XO_SHI_RO_256_PLUS       1,0,0,0,0             0          0,0,0,0,0             0
> XO_SHI_RO_256_SS         0,0,0,0,1             0          0,1,0,2,1             0
> XO_SHI_RO_512_PLUS       0,2,0,0,0             0          0,0,0,2,2             0
> XO_SHI_RO_512_SS         0,0,0,0,0             0          0,1,0,1,0             0
> PCG_XSH_RR_32            0,0,0,0,0             0          0,0,0,0,0             0
> PCG_XSH_RS_32            0,1,2,1,0             0          1,0,1,1,0             0
> PCG_RXS_M_XS_64          0,1,0,0,0             0          0,0,1,0,0             0
> PCG_MCG_XSH_RR_32        0,0,0,0,0             0          0,0,0,0,0             0
> PCG_MCG_XSH_RS_32        2,1,0,1,0             0          1,1,0,0,0             0
> MSWS                     0,0,0,1,2             0          0,0,0,1,0             0
> SFC_32                   0,0,0,1,1             0          0,1,0,0,2             0
> SFC_64                   0,0,0,1,2             0          0,1,0,0,1             0
> JSF_32                   0,0,0,1,2             0          0,0,0,1,1             0
> JSF_64                   0,0,2,0,0             0          1,1,0,0,0             0
> XO_SHI_RO_128_PP         0,0,0,1,1             0          1,0,0,1,0             0
> XO_RO_SHI_RO_128_PP      0,0,0,0,0             0          0,1,0,1,0             0
> XO_SHI_RO_256_PP         0,1,1,0,1             0          0,0,0,0,1             0
> XO_SHI_RO_512_PP         0,0,1,0,0             0          0,0,2,1,0             0
> XO_RO_SHI_RO_1024_PP     0,0,3,0,1             0          0,0,0,0,0             0
> XO_RO_SHI_RO_1024_S      0,1,0,0,0             0          0,0,0,1,0             0
> XO_RO_SHI_RO_1024_SS     0,1,0,0,0             0          0,1,0,0,1             0
>
> Notable changes:
>
> JDK                : 74,72,74,73,74 -> 50,51,52,49,51
> TWO_CMRES          : 4,3,3,5,4      -> 0,1,0,0,0
> XO_SHI_RO_128_PLUS : 1,2,2,1,1      -> 0,0,1,0,0
>
>
> Changes of systematic failures:
>
> JDK                : 70 -> 48
> TWO_CMRES          :  3 ->  0
> XO_SHI_RO_128_PLUS :  1 ->  0
>
>
> Unchanged systematic failures:
>
> WELL_* and MT variants systematically fail linear complexity. WELL_512
> and WELL_1024 also fail matrix rank.
>
> The other generators have similar results before and after, i.e. no
> systematic failures.
>
> Of note the PCG RR variants never fail. This was true in the old results
> too.
>
>
> I've added an option to the results command that generates the table for
> the user guide to add the systematic failures in parentheses after the
> failure counts, e.g.
>
> XO_SHI_RO_512_PP         0,0,1,0,0 (0)
>
> I am not sure if this is worthwhile. There are no cases amongst the good
> generators (no systematic failures) where there are sporadic failures in
> all 5 test runs. Thus all the good generators have at least one run with
> no failures.

Perhaps, you could put in bold the name of good generators
(those with no systematic failure).  That would literally highlight
the recommendation to avoid "java.util.Random".

Regards,
Gilles

>
> So the value would be mainly zero except for the generators noted above.
>
> Alex
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message