spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jaonary Rabarisoa <jaon...@gmail.com>
Subject Re: key class requirement for PairedRDD ?
Date Fri, 17 Oct 2014 07:49:09 GMT
Here what I'm trying to do. My case class is the following :

case class PersonID(id: String, group: String, name: String)

I want to use PersonID as a key in a PairedRDD. But I think the default
equal function don't fit to my need because two PersonID("a","a","a") are
not the same. When I use a tuple (String, String, String) as a key it's
working.

On Fri, Oct 17, 2014 at 9:03 AM, Sonal Goyal <sonalgoyal4@gmail.com> wrote:

> We use our custom classes which are Serializable and have well defined
> hashcode and equals methods through the Java API. Whats the issue you are
> getting?
>
> Best Regards,
> Sonal
> Nube Technologies <http://www.nubetech.co>
>
> <http://in.linkedin.com/in/sonalgoyal>
>
>
>
> On Fri, Oct 17, 2014 at 12:28 PM, Jaonary Rabarisoa <jaonary@gmail.com>
> wrote:
>
>> Dear all,
>>
>> Is it possible to use any kind of object as key in a PairedRDD. When I
>> use a case class  key, the groupByKey operation don't behave as I expected.
>> I want to use a case class to avoid using a large tuple as it is easier to
>> manipulate.
>>
>>
>> Cheers,
>>
>> Jaonary
>>
>
>

Mime
View raw message