spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robin East <robin.e...@xense.co.uk>
Subject Re: [MLLIB] Anyone tried correlation with RDD[Vector] ?
Date Thu, 23 Jul 2015 15:25:58 GMT
The OP’s problem is he gets this:

<console>:47: error: type mismatch;
 found   : org.apache.spark.rdd.RDD[org.apache.spark.mllib.linalg.DenseVector]
 required: org.apache.spark.rdd.RDD[org.apache.spark.mllib.linalg.Vector]
Note: org.apache.spark.mllib.linalg.DenseVector <: org.apache.spark.mllib.linalg.Vector,
but class RDD is invariant in type T.
You may wish to define T as +T instead. (SLS 4.5)

The solution is to ensure you have a RDD[Vector] not RDD[DenseVector]

> On 23 Jul 2015, at 15:30, Rishi Yadav <rishi@infoobjects.com> wrote:
> 
> can you explain what transformation is failing. Here's a simple example.
> 
> http://www.infoobjects.com/spark-calculating-correlation-using-rdd-of-vectors/ <http://www.infoobjects.com/spark-calculating-correlation-using-rdd-of-vectors/>
> 
> On Thu, Jul 23, 2015 at 5:37 AM, <Saif.A.Ellafi@wellsfargo.com <mailto:Saif.A.Ellafi@wellsfargo.com>>
wrote:
> I tried with a RDD[DenseVector] but RDDs are not transformable, so T+ RDD[DenseVector]
not >: RDD[Vector] and can’t get to use the RDD input method of correlation.
>  
> Thanks,
> Saif
>  
> 


Mime
View raw message