spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Darabos <>
Subject Re: What is the best way to handle transformations or actions that takes forever?
Date Tue, 17 Jun 2014 18:34:32 GMT
I think you need to implement a timeout in your code. As far as I know,
Spark will not interrupt the execution of your code as long as the driver
is connected. Might be an idea though.

On Tue, Jun 17, 2014 at 7:54 PM, Peng Cheng <> wrote:

> I've tried enabling the speculative jobs, this seems partially solved the
> problem, however I'm not sure if it can handle large-scale situations as it
> only start when 75% of the job is finished.
> --
> View this message in context:
> Sent from the Apache Spark User List mailing list archive at

View raw message