spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: Kryo registration for Tuples?
Date Thu, 09 Jun 2016 01:00:43 GMT
Yes you can :)


On Wed, Jun 8, 2016 at 6:00 PM, Alexander Pivovarov <apivovarov@gmail.com>
wrote:

> Can I just enable spark.kryo.registrationRequired and look at error
> messages to get unregistered classes?
>
> On Wed, Jun 8, 2016 at 5:52 PM, Reynold Xin <rxin@databricks.com> wrote:
>
>> Due to type erasure they have no difference, although watch out for Scala
>> tuple serialization.
>>
>>
>> On Wednesday, June 8, 2016, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>>> I think the second group (3 classOf's) should be used.
>>>
>>> Cheers
>>>
>>> On Wed, Jun 8, 2016 at 4:53 PM, Alexander Pivovarov <
>>> apivovarov@gmail.com> wrote:
>>>
>>>> if my RDD is RDD[(String, (Long, MyClass))]
>>>>
>>>> Do I need to register
>>>>
>>>> classOf[MyClass]
>>>> classOf[(Any, Any)]
>>>>
>>>> or
>>>>
>>>> classOf[MyClass]
>>>> classOf[(Long, MyClass)]
>>>> classOf[(String, (Long, MyClass))]
>>>>
>>>> ?
>>>>
>>>>
>>>
>

Mime
View raw message