spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: Increase Timeout or optimize Spark UT?
Date Mon, 21 Aug 2017 00:53:10 GMT
It seems like it's time to look into how to cut down some of the test
runtimes. Test runtimes will slowly go up given the way development
happens. 3 hr is already a very long time for tests to run.


On Sun, Aug 20, 2017 at 5:45 PM, Dong Joon Hyun <dhyun@hortonworks.com>
wrote:

> Hi, All.
>
>
>
> Recently, Apache Spark master branch test (SBT with hadoop-2.7 / 2.6) has
> been hitting the build timeout.
>
>
>
> Please see the build time trend.
>
>
>
> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%
> 20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-2.7/buildTimeTrend
>
>
>
> All recent 22 builds fail due to timeout directly/indirectly. The last
> success (SBT with Hadoop-2.7) is 15th August.
>
>
>
> We may do the followings.
>
>
>
>    1. Increase Build Timeout (3 hr 30 min)
>    2. Optimize UTs (Scala/Java/Python/UT)
>
>
>
> But, Option 1 will be the immediate solution for now . Could you update
> the Jenkins setup?
>
>
>
> Bests,
>
> Dongjoon.
>

Mime
View raw message