spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <>
Subject RE: [MLLIB] Anyone tried correlation with RDD[Vector] ?
Date Thu, 23 Jul 2015 19:37:23 GMT
Thank you very much,

working fine so far

From: Robin East []
Sent: Thursday, July 23, 2015 12:26 PM
To: Rishi Yadav
Cc: Ellafi, Saif A.;; Liu, Weicheng
Subject: Re: [MLLIB] Anyone tried correlation with RDD[Vector] ?

The OP’s problem is he gets this:

<console>:47: error: type mismatch;
 found   : org.apache.spark.rdd.RDD[org.apache.spark.mllib.linalg.DenseVector]
 required: org.apache.spark.rdd.RDD[org.apache.spark.mllib.linalg.Vector]
Note: org.apache.spark.mllib.linalg.DenseVector <: org.apache.spark.mllib.linalg.Vector,
but class RDD is invariant in type T.
You may wish to define T as +T instead. (SLS 4.5)

The solution is to ensure you have a RDD[Vector] not RDD[DenseVector]

On 23 Jul 2015, at 15:30, Rishi Yadav <<>>

can you explain what transformation is failing. Here's a simple example.

On Thu, Jul 23, 2015 at 5:37 AM, <<>>
I tried with a RDD[DenseVector] but RDDs are not transformable, so T+ RDD[DenseVector] not
>: RDD[Vector] and can’t get to use the RDD input method of correlation.


View raw message