spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicholas R. Peterson" <nrpeter...@gmail.com>
Subject Spark on Yarn: Kryo throws ClassNotFoundException for class included in fat jar
Date Tue, 08 Sep 2015 03:38:46 GMT
I'm trying to run a Spark 1.4.1 job on my CDH5.4 cluster, through Yarn.
Serialization is set to use Kryo.

I have a large object which I send to the executors as a Broadcast. The
object seems to serialize just fine. When it attempts to deserialize,
though, Kryo throws a ClassNotFoundException... for a class that I include
in the fat jar that I spark-submit.

What could be causing this classpath issue with Kryo on the executors?
Where should I even start looking to try to diagnose the problem? I
appreciate any help you can provide.

Thank you!

-- Nick

Mime
View raw message