spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeyaraj, Arockia R (Arockia)" <arockia.r.jeya...@verizon.com>
Subject RE: Building spark with native library support
Date Thu, 06 Mar 2014 18:03:14 GMT
Hi,

I am trying to setup Spark in windows for development environment. I get following error when
I run sbt. Pl help me to resolve this issue. I am working for Verizon and am in my company
network and can't access internet without proxy.

C:\Users>sbt
Getting org.fusesource.jansi jansi 1.11 ...
You probably access the destination server through a proxy server that is not we
ll configured.
You probably access the destination server through a proxy server that is not we
ll configured.
You probably access the destination server through a proxy server that is not we
ll configured.

:: problems summary ::
:::: WARNINGS
        Host repo.typesafe.com not found. url=http://repo.typesafe.com/typesafe/
ivy-releases/org.fusesource.jansi/jansi/1.11/ivys/ivy.xml

        Host repo1.maven.org not found. url=http://repo1.maven.org/maven2/org/fu
sesource/jansi/jansi/1.11/jansi-1.11.pom

        Host repo1.maven.org not found. url=http://repo1.maven.org/maven2/org/fu
sesource/jansi/jansi/1.11/jansi-1.11.jar

                module not found: org.fusesource.jansi#jansi;1.11

        ==== local: tried

          C:\Users\v983654\.ivy2\local\org.fusesource.jansi\jansi\1.11\ivys\ivy.
xml

          -- artifact org.fusesource.jansi#jansi;1.11!jansi.jar:

          C:\Users\v983654\.ivy2\local\org.fusesource.jansi\jansi\1.11\jars\jans
i.jar

        ==== typesafe-ivy-releases: tried

          http://repo.typesafe.com/typesafe/ivy-releases/org.fusesource.jansi/ja
nsi/1.11/ivys/ivy.xml

        ==== Maven Central: tried

          http://repo1.maven.org/maven2/org/fusesource/jansi/jansi/1.11/jansi-1.
11.pom

          -- artifact org.fusesource.jansi#jansi;1.11!jansi.jar:

          http://repo1.maven.org/maven2/org/fusesource/jansi/jansi/1.11/jansi-1.
11.jar

                ::::::::::::::::::::::::::::::::::::::::::::::

                ::          UNRESOLVED DEPENDENCIES         ::

                ::::::::::::::::::::::::::::::::::::::::::::::

                :: org.fusesource.jansi#jansi;1.11: not found

                ::::::::::::::::::::::::::::::::::::::::::::::



:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
unresolved dependency: org.fusesource.jansi#jansi;1.11: not found
Error during sbt execution: Error retrieving required libraries
  (see C:\Users\v983654\.sbt\boot\update.log for complete log)
Error: Could not retrieve jansi 1.11

Thanks
Arockia Raja

-----Original Message-----
From: Matei Zaharia [mailto:matei.zaharia@gmail.com] 
Sent: Thursday, March 06, 2014 11:44 AM
To: user@spark.apache.org
Subject: Re: Building spark with native library support

Is it an error, or just a warning? In any case, you need to get those libraries from a build
of Hadoop for your platform. Then add them to the SPARK_LIBRARY_PATH environment variable
in conf/spark-env.sh, or to your -Djava.library.path if launching an application separately.

These libraries just speed up some compression codecs BTW, so it should be fine to run without
them too.

Matei

On Mar 6, 2014, at 9:04 AM, Alan Burlison <Alan.Burlison@oracle.com> wrote:

> Hi,
> 
> I've successfully built 0.9.0-incubating on Solaris using sbt, following the instructions
at http://spark.incubator.apache.org/docs/latest/ and it seems to work OK. However, when I
start it up I get an error about missing Hadoop native libraries. I can't find any mention
of how to build the native components in the instructions, how is that done?
> 
> Thanks,
> 
> -- 
> Alan Burlison
> --


Mime
View raw message