How about pointing LD_LIBRARY_PATH to native lib folder ?You need Spark 1.2.0 or higher for the above to work. See SPARK-1719CheersOn Sun, Mar 22, 2015 at 4:02 AM, Xi Shen <email@example.com> wrote:Hi Ted,
I have tried to invoke the command from both cygwin environment and powershell environment. I still get the messages:15/03/22 21:56:00 WARN netlib.BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS15/03/22 21:56:00 WARN netlib.BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLASFrom the Spark UI, I can see:
Thanks,DavidOn Sun, Mar 22, 2015 at 11:45 AM Ted Yu <firstname.lastname@example.org> wrote:Can you try the --driver-library-path option ?spark-submit --driver-library-path /opt/hadoop/lib/native ...CheersOn Sat, Mar 21, 2015 at 4:58 PM, Xi Shen <email@example.com> wrote:Hi,
I use the OpenBLAS DLL, and have configured my application to work in IDE. When I start my Spark application from IntelliJ IDE, I can see in the log that the native lib is loaded successfully.
But if I use spark-submit to start my application, the native lib still cannot be load. I saw the WARN message that it failed to load both the native and native-ref library. I checked the Environment tab in the Spark UI, and the java.library.path is set correctly.