spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 颜发才(Yan Facai) <yaf...@gmail.com>
Subject Spark 2.0.2, KyroSerializer, double[] is not registered.
Date Sun, 08 Jan 2017 05:21:51 GMT
Hi, all.
I enable kyro in spark with spark-defaults.conf:
     spark.serializer
org.apache.spark.serializer.KryoSerializer
     spark.kryo.registrationRequired  true

A KryoException is raised when a logistic regression algorithm is running:
     Note: To register this class use: kryo.register(double[].class);
     Serialization trace:
     currL1 (org.apache.spark.mllib.stat.MultivariateOnlineSummarizer)
            at
com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:585)
            at
com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
            at
com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568)
            at
com.twitter.chill.Tuple2Serializer.write(TupleSerializers.scala:36)
            at
com.twitter.chill.Tuple2Serializer.write(TupleSerializers.scala:33)
           at
com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568)

My question is:
Doesn't double[].class be supported by default?

Thanks.

Mime
View raw message