spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koert Kuipers <ko...@tresata.com>
Subject objectFile uses only java serializer?
Date Mon, 23 Mar 2015 20:14:59 GMT
in the comments on SparkContext.objectFile it says:
"It will also be pretty slow if you use the default serializer (Java
serialization)"

this suggests the spark.serializer is used, which means i can switch to the
much faster kryo serializer. however when i look at the code it uses
Utils.deserialize, which is always using Java serialization.

did i get that right? and is this desired?
it seems straightforward to switch objectFile to use the serializer as
specified by spark.serializer (although it might being in new classloader
issues).

Mime
View raw message