spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Startin <richardstar...@outlook.com>
Subject Re: Kryo On Spark 1.6.0
Date Tue, 10 Jan 2017 11:17:37 GMT
Hi Enrico,


Only set spark.kryo.registrationRequired if you want to forbid any classes you have not explicitly
registered - see http://spark.apache.org/docs/latest/configuration.html.

Configuration - Spark 2.0.2 Documentation<http://spark.apache.org/docs/latest/configuration.html>
spark.apache.org
Spark Configuration. Spark Properties. Dynamically Loading Spark Properties; Viewing Spark
Properties; Available Properties. Application Properties; Runtime Environment

To enable kryo, you just need spark.serializer=org.apache.spark.serializer.KryoSerializer.
There is some info here - http://spark.apache.org/docs/latest/tuning.html

<http://spark.apache.org/docs/latest/tuning.html>Cheers,
Richard



https://richardstartin.com/


________________________________
From: Enrico DUrso <enrico.durso@everis.com>
Sent: 10 January 2017 11:10
To: user@spark.apache.org
Subject: Kryo On Spark 1.6.0


Hi,

I am trying to use Kryo on Spark 1.6.0.
I am able to register my own classes and it works, but when I set "spark.kryo.registrationRequired
" to true, I get an error about a scala class:
"Class is not registered: scala.collection.mutable.WrappedArray$ofRef".

Any of you has already solved this issue in Java? I found the code to solve it in Scala, but
unable to register this class in Java.

Cheers,

enrico

________________________________

CONFIDENTIALITY WARNING.
This message and the information contained in or attached to it are private and confidential
and intended exclusively for the addressee. everis informs to whom it may receive it in error
that it contains privileged information and its use, copy, reproduction or distribution is
prohibited. If you are not an intended recipient of this E-mail, please notify the sender,
delete it and do not read, act upon, print, disclose, copy, retain or redistribute any portion
of this E-mail.

Mime
View raw message