commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Herbert <alex.d.herb...@gmail.com>
Subject Re: [Rng] New XoShiRo generators
Date Mon, 18 Mar 2019 15:48:59 GMT

On 18/03/2019 14:12, Gilles Sadowski wrote:
> Hi.
>
>>>> [...]
>>>>
>>>> One actual issue is that we are testing long providers using the long to
create 2 int values. Should we test using a series of the upper 32 bits and then a series
of the lower 32 bits?
>>> Is that useful since the test now sees the integers as they are produced (i.e.
2
>>> values per long)?
>>>
>> It is not relevant if you are concerned about int quality. But if you are concerned
about long quality then it is relevant. The long quality is important for the quality of nextDouble().
Although in that case only the upper 53 bits of the long. This means that the quality of a
long from an int provider is also not covered by the benchmark as that would require testing
alternating ints twice using the series: 1, 3, 5…, 2n+1 and 2, 4, 6, … 2n.
> I don't follow: I'd think that if the full sequence passes the test,
> then "decimated"
> sequences will too.

My position was that if a series of int values is random, that does not 
mean a subset of the int values is random due to bias in the subset sample.

However I acknowledge that:

- the test suites may have this covered already

- if it really is random then any subset will also be random, even if it 
is a systematic subset such as alternating values

>> Given that half of the int values were previously discarded from the BigCrush analysis,
the current results on the user guide page actually represent BigCrush running on the upper
32-bits of the long, byte reversed due to the big/little endian interpretation of the bytes
in Java and linux.
>>
>> So maybe the an update to the RandomStressTester to support analysis for int or long
quality is needed.
> I'm not convinced.

I'm not totally convinced either. It is a lot more work to test upper 
and lower bits separately.

It may be that a producer of long values has better randomness in the 
upper bits. Or put another way has less than 64-bits of randomness.

The question is whether running the test suite on all the bits (as we 
currently do) or targetting just the upper or lower 32-bits is useful. 
E.g. would a RNG that fails a few tests using all the bits pass with 
just the upper 32-bits and fail more with just the lower 32-bits, or 
would the fails be the same?

Note: The current results for long providers do not test the lower 
32-bits at all, and currently test alternating values from any int 
providers. So they will have to be rerun anyway.

Previously I looked at systematic failures in the test suite (where the 
same test always fails). IIRC the MersenneTwister has some systematic 
failures. Since we are not doing systematic failure analysis for the 
user guide, and we are not developing the algorithms, then I agree that 
a more detailed analysis of the failures and their origins is beyond the 
scope of the quality section.

So leave the testing to just ints and document on the user guide that is 
what we are testing.

>> For now the quality section on the website should just state that the quality is
for the ‘nextInt()’ method of the RNG.
>>
>> I have the results of BigCrush using the new bridge c program:
>>
>> XorShiftSerialComposite : 40, 39, 39 : 608.2 +/- 3.9
> Makes sense now. :-}
>
>> So it fails.
>>
>> The XorShiftXorComposite crashed after 2 hours about 1/4 of the results file complete.
I am running again so I can monitor it for memory usage. Something in the BigCrush suite just
cannot handle this generator output.
> Strange...

Yep. I restarted it and it crashed after 3 hours again! Monitoring every 
minute found no obvious memory issues. The BigCrush process never 
exceeded 2.7% of memory and Java never exceeded 0.1%.

The footer is written by the Java program so this indicates that the 
TestU01 bridge is stopping then the Java process writes the footer, 
wraps everything up and stops.

Weirdly my process to follow the output also stopped which is 
unexpected. I am investigating if my system has some strange walltime 
limits I do not know about. Since the other composite generators work 
using the same code I am thinking it may be a bug in TestU01 when the 
generator is bad (which DieHarder thinks it definitely is). But I am 
prepared to be wrong on that and also to never find out.

I've changed RandomStressTester to redirect the stderr to stdout (in 
case that contains any info) and added a line to get the exit code from 
the Java Process that is running BigCrush. Maybe that will be non-zero. 
So I re-run and wait 3+ hours again...

Alex



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message