lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Weiss <>
Subject Re: Question on -Dtests.iters needing the glob
Date Sun, 09 Dec 2012 13:52:07 GMT
> So now I'm even more confused. What is tests.dups good for then? Just running the same
suite multiple times, but not changing the seed?

Yes. This was originally requested by Mark (I believe) -- we had a
non-deterministic test for which many repetitions were needed. Because
tests.iters cannot be parallelised (they're a sequence of tests under
a single suite) we duplicated the entire suites -- these are
independent and could be distributed across forked jvms.

> Dawid, can tests.dups use different seeds? Then it'd be really useful cause it can run
these iters in parallel.

No, not at the moment. I do realize it's a pain and I have it in mind
that it'd be great to have such a possibility but it's conflicting
with the architecture of the runner right now. I don't have a clear
vision how to solve this but I know it's not going to be trivial.

There is an issue, check out the discussion:

in particular this one:

> -Dtests.iters generate different seeds for each suite/run. It's good if you
> have a random test and want to test its (or the code's) robustness.

tests.iters just multiplies every test N times. Whether it gets a
constant seed or a derivative seed depends on annotations and/or
-Dtests.seed override. By default what you say is true but try running
your tests with:

-Dtests.iters=10 -Dtests.seed=deadbeef:deadf00d

and you'll see what I mean.

> -Dtests.dups uses the same seed for every suite, but can run in parallel JVMs.

tests.dups is essentially feeding the same class name to the runner
twice. There is one master seed so both classes should run
IDENTICALLY. You can speed up stress testing if you have multiple
cores or if you wish to run the same test over and over (even with a
single JVM) but it's going to be the same test (unless it's
non-deterministic or in other way does not depend on the seed).

> But I'm sure Dawid has thought about it, and there's some JUnit limitation
> that does not allow it :).

This time it's not even JUnit but the code I wrote -- it strictly
follows the principle of a single master seed, it worked great for
other things but I didn't think of repeating the same suite many
times, each time with a different seed and it's hard to integrate it
right now.

I'm sure there's a way to rewrite the code, I just didn't have the
time to look into it -- lots of stuff piling up, sorry.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message