Thanks for the discussion. I followed up on your problem and learned that Scala doesn't support multiple implicit conversions in a single expression for complexity reasons. I'm afraid the solution for now is to do (v1: BV[Double]) + (v1: BV[Double])

On Tue, Aug 25, 2015 at 11:06 AM, Kristina Rogale Plazonic <> wrote:


On Tue, Aug 25, 2015 at 1:57 PM, Burak Yavuz <> wrote:
Hmm. I have a lot of code on the local linear algebra operations using Spark's Matrix and Vector representations

I can make a Spark package with that code if people are interested.


On Tue, Aug 25, 2015 at 10:54 AM, Kristina Rogale Plazonic <> wrote:
However I do think it's easier than it seems to write the implicits;
it doesn't involve new classes or anything. Yes it's pretty much just
what you wrote. There is a class "Vector" in Spark. This declaration
can be in an object; you don't implement your own class. (Also you can
use "toBreeze" to get Breeze vectors.)

The implicit conversion with the implicit def happens for the first vector in the sum, but not the second vector (see below). 

At this point I give up, because I spent way too much time.  I am so disappointed.  So many times I heard "Spark makes simple things easy and complicated things possible". Well, here is the simplest thing you can imagine in linear algebra, but heck, it is not easy or intuitive.  It was easier to run a DeepLearning algo (from another library) than add two vectors. 

If anybody has a workaround other than implementing your own add/substract/scalarMultiply, PLEASE let me know. 

Here is the code and error from (freshly started) spark-shell: 

scala> import breeze.linalg.{DenseVector => BDV, SparseVector => BSV, Vector => BV}
import breeze.linalg.{DenseVector=>BDV, SparseVector=>BSV, Vector=>BV}

scala> import org.apache.spark.mllib.linalg.Vectors
import org.apache.spark.mllib.linalg.Vectors

scala> val v1 = Vectors.dense(1.0, 2.0, 3.0)
v1: org.apache.spark.mllib.linalg.Vector = [1.0,2.0,3.0]

scala> import org.apache.spark.mllib.linalg.{Vector =>SparkVector}
import org.apache.spark.mllib.linalg.{Vector=>SparkVector}

scala> object MyUtils {
     |   implicit def toBreeze(v:SparkVector) = BV(v.toArray)
     | }
warning: there were 1 feature warning(s); re-run with -feature for details
defined module MyUtils

scala> import MyUtils._
import MyUtils._

scala> v1:BV[Double]
res2: breeze.linalg.Vector[Double] = DenseVector(1.0, 2.0, 3.0)

scala> v1 + v1
<console>:30: error: could not find implicit value for parameter op: breeze.linalg.operators.OpAdd.Impl2[breeze.linalg.Vector[Double],org.apache.spark.mllib.linalg.Vector,That]
              v1 + v1