spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: foreachRDD causing executor lost failure
Date Wed, 09 Sep 2015 07:00:48 GMT
If you can look a bit in the executor logs, you would see the exact reason
(mostly a OOM/GC etc). Instead of using foreach, try to use mapPartitions
or foreachPartitions.

Thanks
Best Regards

On Tue, Sep 8, 2015 at 10:45 PM, Priya Ch <learnings.chitturi@gmail.com>
wrote:

> Hello All,
>
>  I am using foreachRDD in my code as -
>
>   dstream.foreachRDD { rdd => rdd.foreach { record => // look up with
> cassandra table
> // save updated rows to cassandra table.
> }
> }
>  This foreachRDD is causing executor lost failure. what is the behavior of
> this foreachRDD ???
>
> Thanks,
> Padma Ch
>

Mime
View raw message