bq. it uses Utils.deserialize, which is always using Java serialization. 

I agree with your finding.

On Mon, Mar 23, 2015 at 1:14 PM, Koert Kuipers <koert@tresata.com> wrote:
in the comments on SparkContext.objectFile it says:
"It will also be pretty slow if you use the default serializer (Java serialization)"

this suggests the spark.serializer is used, which means i can switch to the much faster kryo serializer. however when i look at the code it uses Utils.deserialize, which is always using Java serialization.

did i get that right? and is this desired?
it seems straightforward to switch objectFile to use the serializer as specified by spark.serializer (although it might being in new classloader issues).