spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Pfeiffer <...@preferred.jp>
Subject Re: Why must the dstream.foreachRDD(...) parameter be serializable?
Date Wed, 28 Jan 2015 02:38:42 GMT
Hi,

thanks for the answers!

On Wed, Jan 28, 2015 at 11:31 AM, Shao, Saisai <saisai.shao@intel.com>
wrote:
>
> Also this `foreachFunc` is more like an action function of RDD, thinking
> of rdd.foreach(func), in which `func` need to be serializable. So maybe I
> think your way of use it is not a normal way :).
>

Yeah I totally understand why func in rdd.foreach(func) must be
serializable (because it's sent to the executors), but I didn't get why a
function that's not shipped around must be serializable, too.

The explanations made sense, though :-)

Thanks
Tobias

Mime
View raw message