spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: How Spark handles dead machines during a job.
Date Sat, 09 Apr 2016 06:35:31 GMT
The driver has the data and wouldn't need to rerun.

On Friday, April 8, 2016, Sung Hwan Chung <codedeft@gmail.com> wrote:

> Hello,
>
> Say, that I'm doing a simple rdd.map followed by collect. Say, also, that
> one of the executors finish all of its tasks, but there are still other
> executors running.
>
> If the machine that hosted the finished executor gets terminated, does the
> master still have the results from the finished tasks (and thus doesn't
> restart those finished tasks)?
>
> Or does the master require that all the executors be alive during the
> entire map-collect cycle?
>
> Thanks!
>

Mime
View raw message