Did you build Spark with: -Pnetlib-lgpl?

Ref: https://spark.apache.org/docs/latest/mllib-guide.html

Burak

On Sun, Mar 22, 2015 at 7:37 AM, Ted Yu <yuzhihong@gmail.com> wrote:
How about pointing LD_LIBRARY_PATH to native lib folder ?

You need Spark 1.2.0 or higher for the above to work. See SPARK-1719

Cheers

On Sun, Mar 22, 2015 at 4:02 AM, Xi Shen <davidshen84@gmail.com> wrote:
Hi Ted,

I have tried to invoke the command from both cygwin environment and powershell environment. I still get the messages:

15/03/22 21:56:00 WARN netlib.BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS
15/03/22 21:56:00 WARN netlib.BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS

From the Spark UI, I can see:

  spark.driver.extraLibrary c:\openblas


Thanks,
David


On Sun, Mar 22, 2015 at 11:45 AM Ted Yu <yuzhihong@gmail.com> wrote:
Can you try the --driver-library-path option ?

spark-submit --driver-library-path /opt/hadoop/lib/native ...

Cheers

On Sat, Mar 21, 2015 at 4:58 PM, Xi Shen <davidshen84@gmail.com> wrote:
Hi,

I use the OpenBLAS DLL, and have configured my application to work in IDE. When I start my Spark application from IntelliJ IDE, I can see in the log that the native lib is loaded successfully.

But if I use spark-submit to start my application, the native lib still cannot be load. I saw the WARN message that it failed to load both the native and native-ref library. I checked the Environment tab in the Spark UI, and the java.library.path is set correctly.


Thanks,

David