spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiangrui Meng <>
Subject Re: MLLIB usage: BLAS dependency warning
Date Wed, 12 Nov 2014 06:47:29 GMT
Could you try "jar tf" on the assembly jar and grep
""? -Xiangrui

On Tue, Nov 11, 2014 at 7:11 PM, jpl <> wrote:
> Hi,
> I am having trouble using the BLAS libs with the MLLib functions.  I am
> using org.apache.spark.mllib.clustering.KMeans (on a single machine) and
> running the Spark-shell with the kmeans example code (from
>  which runs
> successfully but I get the following warning in the log:
> WARN netlib.BLAS: Failed to load implementation from:
> com.github.fommil.netlib.NativeSystemBLAS
> WARN netlib.BLAS: Failed to load implementation from:
> com.github.fommil.netlib.NativeRefBLAS
> I compiled spark 1.1.0 with mvn -Phadoop-2.4  -Dhadoop.version=2.4.0
> -Pnetlib-lgpl -DskipTests clean package
> If anyone could please clarify the steps to get the dependencies correctly
> installed and visible to spark (from
>, that would be
> greatly appreciated.  Using yum, I installed blas.x86_64, lapack.x86_64,
> gcc-gfortran.x86_64, libgfortran.x86_64 and then downloaded Breeze and built
> that successfully with Maven.  I verified that I do have
> /usr/lib/ and /usr/lib/ present on the machine and
> ldconf -p shows these listed.
> I also tried adding /usr/lib/ to spark.executor.extraLibraryPath and I
> verified it is present in the Spark webUI environment tab.   I downloaded
> and compiled jblas with mvn clean install, which creates
> jblas-1.2.4-SNAPSHOT.jar, and then also tried adding that to
> spark.executor.extraClassPath but I still get the same WARN message. Maybe
> there are a few simple steps that I am missing?  Thanks a lot.
> --
> View this message in context:
> Sent from the Apache Spark User List mailing list archive at
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message