lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <>
Subject [jira] [Commented] (LUCENE-4463) add support for running the same test method many times
Date Sat, 06 Oct 2012 07:02:03 GMT


Robert Muir commented on LUCENE-4463:

if you can settle for reusing the same JVM, tests.iters combined with testmethod (or tests.method=...*)
will give you different test seeds everytime – only the global seed will be the same 

Right, we can't really settle for that for Lucene's tests. Thats because things like Codec
are set at class level, so i could run this 100000000000000 times and press commit and watch
it fail because jenkins gets a different codec. And we have a lot of these, and only more
being added.

Its a tradeoff, sure we could set Codec per-writer e.g. in our newIndexWriterConfig instead
of per-test-class, but I think it makes debugging much simpler to look at it as a "parameterized-test-class"
at this level of Codec apis maturity so we can easily see which one the test got when it failed.

So really we need a different per-class seed too: same as you would get when doing 'ant test'
in a loop with an inefficient shell script.

> add support for running the same test method many times
> -------------------------------------------------------
>                 Key: LUCENE-4463
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Wish
>          Components: general/build
>            Reporter: Robert Muir
>         Attachments: LUCENE-4463.patch
> I have a shell script for this, mike has a python script, its annoying :)
> I want to do something like this:
> ant beast -Dtestcase=XXXX -Dtestmethod=YYYY -Diterations=100
> I would be happy with a simple loop that just invokes 'test' somehow: getting a fresh
new JVM to each iteration is desirable anyway (so you get fresh codecs, etc). 
> the -Dtests.iters is not really useful for this because it does not allow -Dtestmethod
and it does not give a fresh jvm.
> bonus points if it can use multiple jvms at the same time though :)

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message