commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Herbert <alex.d.herb...@gmail.com>
Subject Re: [rng] BigCrush results
Date Fri, 11 Oct 2019 13:54:12 GMT

On 07/10/2019 15:00, Gilles Sadowski wrote:
> Hi.
>
> Le lun. 7 oct. 2019 à 15:54, Alex Herbert <alex.d.herbert@gmail.com> a écrit
:
>> I have found a bug in the stress test application that affects all the
>> BigCrush results.
>>
>> This was found when updating the stress test application to support
>> 64-bit testing within PractRand, and testing of upper and lower 32-bits
>> from 64-bit out. An initial attempt to show that the lower 32-bits, bit
>> reversed, of the XOR_SHIFT_1024_S systematically fail Linear Complexity
>> tests did not work.
>>
>> The bug was a dual reversal of the platform byte-order. Code was added
>> to reverse the byte order in the link c application that read from stdin
>> and passes numbers to BigCrush. Later code was added to reverse the byte
>> order in the Java application so allowing Dieharder to work. The
>> original c application should have been updated but was not (despite the
>> fact I thought I had done this). So the byte order was not correct after
>> a dual reversal and the BigCrush results refer to a byte reversed output
>> sequence.
>>
>> I fixed the code and have verified that the stress test application can
>> identify Linear Complexity failures in XOR_SHIFT_1024_S.
>>
>> I am rerunning BigCrush for the other generators.
>>
>> Testing of generators with PractRand is still under trial and will
>> probably not make it for the 1.3 release. The only outstanding item for
>> 1.3 is continued work on updating the user guide.
> The userguide (and web site) can be updated at any time (between
> releases); it is not be a blocker.

BigCrush has finished. Not much changed.

I've added an option to look for systematic failures to the stress test 
application. It is shown in the below table using the intersection 
symbol (view these tables using a fixed width font):

RNG                      New                   ∩     
    Old                   ∩
JDK                      50,51,52,49,51        48        
74,72,74,73,74        70
WELL_512_A               6,7,8,6,6             6         
7,6,6,6,6             6
WELL_1024_A              5,4,5,5,4             4          4,4,5,4,4        
    4
WELL_19937_A             2,2,3,3,3             2          3,3,2,2,2        
    2
WELL_19937_C             4,2,2,2,2             2          2,2,3,2,2        
    2
WELL_44497_A             3,2,2,2,3             2          2,2,2,2,3        
    2
WELL_44497_B             2,2,2,2,2             2          2,3,2,2,2        
    2
MT                       2,3,2,2,3             2     
    2,3,2,2,2             2
ISAAC                    1,1,0,0,1             0         
0,1,2,0,0             0
SPLIT_MIX_64             0,0,0,1,0             0          1,0,0,0,0        
    0
XOR_SHIFT_1024_S         1,0,0,0,2             0          0,0,0,0,0        
    0
TWO_CMRES                0,1,0,0,0             0         
4,3,3,5,4             3
MT_64                    2,2,2,2,2             2         
2,3,2,2,2             2
MWC_256                  0,1,1,1,0             0         
0,0,0,2,0             0
KISS                     1,1,0,0,0             0         
0,0,0,0,0             0
XOR_SHIFT_1024_S_PHI     0,2,0,0,1             0          0,0,0,0,0        
    0
XO_RO_SHI_RO_64_S        1,2,3,1,1             1          1,1,2,1,3        
    1
XO_RO_SHI_RO_64_SS       0,1,0,0,0             0          0,0,0,0,0        
    0
XO_SHI_RO_128_PLUS       0,0,1,0,0             0          1,2,2,1,1        
    1
XO_SHI_RO_128_SS         1,0,1,0,0             0          0,1,0,0,0        
    0
XO_RO_SHI_RO_128_PLUS    1,0,0,0,0             0          0,1,0,0,0        
    0
XO_RO_SHI_RO_128_SS      1,1,1,0,0             0          1,0,1,0,0        
    0
XO_SHI_RO_256_PLUS       1,0,0,0,0             0          0,0,0,0,0        
    0
XO_SHI_RO_256_SS         0,0,0,0,1             0          0,1,0,2,1        
    0
XO_SHI_RO_512_PLUS       0,2,0,0,0             0          0,0,0,2,2        
    0
XO_SHI_RO_512_SS         0,0,0,0,0             0          0,1,0,1,0        
    0
PCG_XSH_RR_32            0,0,0,0,0             0          0,0,0,0,0        
    0
PCG_XSH_RS_32            0,1,2,1,0             0          1,0,1,1,0        
    0
PCG_RXS_M_XS_64          0,1,0,0,0             0          0,0,1,0,0        
    0
PCG_MCG_XSH_RR_32        0,0,0,0,0             0          0,0,0,0,0        
    0
PCG_MCG_XSH_RS_32        2,1,0,1,0             0          1,1,0,0,0        
    0
MSWS                     0,0,0,1,2             0         
0,0,0,1,0             0
SFC_32                   0,0,0,1,1             0         
0,1,0,0,2             0
SFC_64                   0,0,0,1,2             0         
0,1,0,0,1             0
JSF_32                   0,0,0,1,2             0         
0,0,0,1,1             0
JSF_64                   0,0,2,0,0             0         
1,1,0,0,0             0
XO_SHI_RO_128_PP         0,0,0,1,1             0          1,0,0,1,0        
    0
XO_RO_SHI_RO_128_PP      0,0,0,0,0             0          0,1,0,1,0        
    0
XO_SHI_RO_256_PP         0,1,1,0,1             0          0,0,0,0,1        
    0
XO_SHI_RO_512_PP         0,0,1,0,0             0          0,0,2,1,0        
    0
XO_RO_SHI_RO_1024_PP     0,0,3,0,1             0          0,0,0,0,0        
    0
XO_RO_SHI_RO_1024_S      0,1,0,0,0             0          0,0,0,1,0        
    0
XO_RO_SHI_RO_1024_SS     0,1,0,0,0             0          0,1,0,0,1        
    0

Notable changes:

JDK                : 74,72,74,73,74 -> 50,51,52,49,51
TWO_CMRES          : 4,3,3,5,4      -> 0,1,0,0,0
XO_SHI_RO_128_PLUS : 1,2,2,1,1      -> 0,0,1,0,0


Changes of systematic failures:

JDK                : 70 -> 48
TWO_CMRES          :  3 ->  0
XO_SHI_RO_128_PLUS :  1 ->  0


Unchanged systematic failures:

WELL_* and MT variants systematically fail linear complexity. WELL_512 
and WELL_1024 also fail matrix rank.

The other generators have similar results before and after, i.e. no 
systematic failures.

Of note the PCG RR variants never fail. This was true in the old results 
too.


I've added an option to the results command that generates the table for 
the user guide to add the systematic failures in parentheses after the 
failure counts, e.g.

XO_SHI_RO_512_PP         0,0,1,0,0 (0)

I am not sure if this is worthwhile. There are no cases amongst the good 
generators (no systematic failures) where there are sporadic failures in 
all 5 test runs. Thus all the good generators have at least one run with 
no failures.

So the value would be mainly zero except for the generators noted above.

Alex


>
> Best regards,
> Gilles
>
>> Alex
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message