spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From waymost <>
Subject Spark attempts to de/serialize using JavaSerializer despite being configured to use Kryo
Date Sun, 18 Jan 2015 01:04:00 GMT
I'm new to Spark and have run into issues using Kryo for serialization
instead of Java. I have my SparkConf configured as such:

val conf = new SparkConf().setMaster("local").setAppName("test")
        .set("spark.serializer", classOf[KryoSerializer].getName)
        .set("spark.kryo.registrator", classOf[TestRegistrator].getName)

However, when I attempt to run my Spark job, I get the following exception
and stack trace: ... no valid constructor
at ...
at ...

If I go into the REPL. use that same SparkConf to create a KryoSerializer
manually, and then do a serialization-deserialization loop on my instance, I
get it returned back to me without an exception.

I'm perplexed as to why Spark isn't using Kryo when running my job. Any help
would be greatly appreciated.


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message