spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frank Austin Nothaft <fnoth...@berkeley.edu>
Subject Re: Kryo fails with avro having Arrays and unions, but succeeds with simple avro.
Date Fri, 19 Sep 2014 13:23:51 GMT
Hi Mohan,

It’s a bit convoluted to follow in their source, but they essentially typedef KSerializer
as being a KryoSerializer, and then their serializers all extend KSerializer. Spark should
identify them properly as Kryo Serializers, but I haven’t tried it myself.

Regards,

Frank Austin Nothaft
fnothaft@berkeley.edu
fnothaft@eecs.berkeley.edu
202-340-0466

On Sep 19, 2014, at 12:03 AM, mohan.gadm <mohan.gadm@gmail.com> wrote:

> Thanks for the info frank.
> Twitter's-chill avro serializer looks great.
> But how does spark identifies it as serializer, as its not extending from
> KryoSerializer.
> (sorry scala is an alien lang for me). 
> 
> 
> 
> -----
> Thanks & Regards,
> Mohan
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Kryo-fails-with-avro-having-Arrays-and-unions-but-succeeds-with-simple-avro-tp14549p14649.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
> 


Mime
View raw message