spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renxia Wang <renxia.w...@gmail.com>
Subject Re: How to Set Retry Policy in Spark
Date Thu, 01 Oct 2015 23:12:07 GMT
Additional Info: I am running Spark on YARN.

2015-10-01 15:42 GMT-07:00 Renxia Wang <renxia.wang@gmail.com>:

> Hi guys,
>
> I know there is a way to set the number of retry of failed tasks, using
> spark.task.maxFailures. what is the default policy for the failed tasks
> retry? Is it exponential backoff? My tasks sometimes failed because of
> Socket connection timeout/reset, even with retry, some of the tasks will
> fail for more than spark.task.maxFailures times.
>
> Thanks,
>
> Zhique
>

Mime
View raw message