spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Armbrust <mich...@databricks.com>
Subject Re: PairRDDFunctions with Tuple2 subclasses
Date Thu, 20 Nov 2014 00:45:22 GMT
I think you should also be able to get away with casting it back and forth
in this case using .asInstanceOf.

On Wed, Nov 19, 2014 at 4:39 PM, Daniel Siegmann <daniel.siegmann@velos.io>
wrote:

> I have a class which is a subclass of Tuple2, and I want to use it with
> PairRDDFunctions. However, I seem to be limited by the invariance of T in
> RDD[T] (see SPARK-1296 <https://issues.apache.org/jira/browse/SPARK-1296>
> ).
>
> My Scala-fu is weak: the only way I could think to make this work would be
> to define my own equivalent of PairRDDFunctions which works with my
> class, does type conversions to Tuple2, and delegates to PairRDDFunctions.
>
> Does anyone know a better way? Anyone know if there will be a significant
> performance penalty with that approach?
>
> --
> Daniel Siegmann, Software Developer
> Velos
> Accelerating Machine Learning
>
> 54 W 40th St, New York, NY 10018
> E: daniel.siegmann@velos.io W: www.velos.io
>

Mime
View raw message