pirk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Darin Johnson <dbjohnson1...@gmail.com>
Subject Re: Distributed tests take a long time, and that's ok.
Date Thu, 06 Oct 2016 22:39:46 GMT
Being pretty familiar with that section of the code base right now.  I
think a few things could be done.  Regardless I'm refactoring it for
submodules do it'll be easier to iterate these. The main place I'd start is
currently each platform test creates a new input, which means the querier
logic is repeated for the number of platforms tested - setting up the test
inputs once, running all platforms for that test and then tearing down the
inputs would be a start.


Maybe have all tests implement an interface like:

Public interface BaseTest {


public void createInputs(...);


public boolean checkOutputs(List results);

}


The a runner could do:

BaseTest myTest=new myTest()

myTest.createInputs(...)

For(responder:responders)

{

results = responder.runDistributedTest();

If(!myTest.checkResults(results) fail();

}


If nothing else it'd make adding tests easier.



On Oct 6, 2016 9:33 AM, "Walter Ray-Dulany" <raydulany@apache.org> wrote:

> Tim Ellison noted in a recent email to the list (subject: [CANCEL] [VOTE]:
> Apache Pirk 0.2.0-incubating Realse) the following:
>
> > The distributed tests took a long time (~35mins IIRC), is that normal?
>
> I feel like noting this in a separate thread, and discussing it here, is
> worthwhile.
>
> The distributed tests *do* take a long time. The reason isn't because any
> one test is slow; indeed, most of the individual tests take only a handful
> of seconds. The slowness is because of the very large number of tests. This
> vast array of tests, upon inspection, can be seen to be caused by a very
> thorough testing of the large number of different actions that can be
> performed by Pirk and platforms it supports.
>
> I think this is ok. As an example of why this thorough testing is
> necessary, just yesterday, while finishing up PIRK-45, I ran into an error
> 15 minutes into the distributed tests. Without this thorough testing, that
> code would have made it to a PR yesterday night, and (without the
> distributed tests) it is likely no one would have caught it (it was a
> subtle "Spark has an ancient package on its classpath pre-empting a newer
> version I'd included, and the old package has a bug" error).
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message