spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mridul Muralidharan <mri...@gmail.com>
Subject Re: Unit tests in < 5 minutes
Date Sat, 09 Aug 2014 13:00:33 GMT
Issue with supporting this imo is the fact that scala-test uses the
same vm for all the tests (surefire plugin supports fork, but
scala-test ignores it iirc).
So different tests would initialize different spark context, and can
potentially step on each others toes.

Regards,
Mridul


On Fri, Aug 8, 2014 at 9:31 PM, Nicholas Chammas
<nicholas.chammas@gmail.com> wrote:
> Howdy,
>
> Do we think it's both feasible and worthwhile to invest in getting our unit
> tests to finish in under 5 minutes (or something similarly brief) when run
> by Jenkins?
>
> Unit tests currently seem to take anywhere from 30 min to 2 hours. As
> people add more tests, I imagine this time will only grow. I think it would
> be better for both contributors and reviewers if they didn't have to wait
> so long for test results; PR reviews would be shorter, if nothing else.
>
> I don't know how how this is normally done, but maybe it wouldn't be too
> much work to get a test cycle to feel lighter.
>
> Most unit tests are independent and can be run concurrently, right? Would
> it make sense to build a given patch on many servers at once and send
> disjoint sets of unit tests to each?
>
> I'd be interested in working on something like that if possible (and
> sensible).
>
> Nick

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message