spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Pfeiffer <>
Subject Re: Why must the dstream.foreachRDD(...) parameter be serializable?
Date Wed, 28 Jan 2015 02:38:42 GMT

thanks for the answers!

On Wed, Jan 28, 2015 at 11:31 AM, Shao, Saisai <>
> Also this `foreachFunc` is more like an action function of RDD, thinking
> of rdd.foreach(func), in which `func` need to be serializable. So maybe I
> think your way of use it is not a normal way :).

Yeah I totally understand why func in rdd.foreach(func) must be
serializable (because it's sent to the executors), but I didn't get why a
function that's not shipped around must be serializable, too.

The explanations made sense, though :-)


View raw message