spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yadid Ayzenberg <ya...@media.mit.edu>
Subject Re: NoSuchMethodError: breeze.linalg.DenseMatrix
Date Sun, 04 May 2014 20:26:52 GMT
An additional option 4) Use SparkContext.addJar() and have the 
application ship your jar to all the nodes.


Yadid

On 5/4/14, 4:07 PM, DB Tsai wrote:
> If you add the breeze dependency in your build.sbt project, it will 
> not be available to all the workers.
>
> There are couple options, 1) use sbt assembly to package breeze into 
> your application jar. 2) manually copy breeze jar into all the nodes, 
> and have them in the classpath. 3) spark 1.0 has breeze jar in the 
> spark flat assembly jar, so you don't need to add breeze dependency 
> yourself.
>
>
> Sincerely,
>
> DB Tsai
> -------------------------------------------------------
> My Blog: https://www.dbtsai.com
> LinkedIn: https://www.linkedin.com/in/dbtsai
>
>
> On Sun, May 4, 2014 at 4:07 AM, wxhsdp <wxhsdp@gmail.com 
> <mailto:wxhsdp@gmail.com>> wrote:
>
>     Hi,
>       i'am trying to use breeze linalg library for matrix operation in
>     my spark
>     code. i already add dependency
>       on breeze in my build.sbt, and package my code sucessfully.
>
>       when i run on local mode, sbt "run local...", everything is ok
>
>       but when turn to standalone mode, sbt "run
>     spark://127.0.0.1:7077...",
>     error occurs
>
>     14/05/04 18:56:29 WARN scheduler.TaskSetManager: Loss was due to
>     java.lang.NoSuchMethodError
>     java.lang.NoSuchMethodError:
>     breeze.linalg.DenseMatrix$.implOpMulMatrix_DMD_DMD_eq_DMD()Lbreeze/linalg/operators/DenseMatrixMultiplyStuff$implOpMulMatrix_DMD_DMD_eq_DMD$;
>
>       in my opinion, everything needed is packaged to the jar file,
>     isn't it?
>       and does anyone used breeze before? is it good for matrix operation?
>
>
>
>     --
>     View this message in context:
>     http://apache-spark-user-list.1001560.n3.nabble.com/NoSuchMethodError-breeze-linalg-DenseMatrix-tp5310.html
>     Sent from the Apache Spark User List mailing list archive at
>     Nabble.com.
>
>


Mime
View raw message