spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Building spark 1.2 from source requires more dependencies
Date Thu, 26 Mar 2015 22:44:23 GMT
Looking at output from dependency:tree, servlet-api is brought in by the
following:

[INFO] +- org.apache.cassandra:cassandra-all:jar:1.2.6:compile
[INFO] |  +- org.antlr:antlr:jar:3.2:compile
[INFO] |  +- com.googlecode.json-simple:json-simple:jar:1.1:compile
[INFO] |  +- org.yaml:snakeyaml:jar:1.6:compile
[INFO] |  +- edu.stanford.ppl:snaptree:jar:0.1:compile
[INFO] |  +- org.mindrot:jbcrypt:jar:0.3m:compile
[INFO] |  +- org.apache.thrift:libthrift:jar:0.7.0:compile
[INFO] |  |  \- javax.servlet:servlet-api:jar:2.5:compile

FYI

On Thu, Mar 26, 2015 at 3:36 PM, Pala M Muthaia <mchettiar@rocketfuelinc.com
> wrote:

> Hi,
>
> We are trying to build spark 1.2 from source (tip of the branch-1.2 at the
> moment). I tried to build spark using the following command:
>
> mvn -U -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -Phive
> -Phive-thriftserver -DskipTests clean package
>
> I encountered various missing class definition exceptions (e.g: class
> javax.servlet.ServletException not found).
>
> I eventually got the build to succeed after adding the following set of
> dependencies to the spark-core's pom.xml:
>
>     <dependency>
>       <groupId>javax.servlet</groupId>
>       <artifactId>*servlet-api*</artifactId>
>       <version>3.0</version>
>     </dependency>
>
>     <dependency>
>       <groupId>org.eclipse.jetty</groupId>
>       <artifactId>*jetty-io*</artifactId>
>     </dependency>
>
>     <dependency>
>       <groupId>org.eclipse.jetty</groupId>
>       <artifactId>*jetty-http*</artifactId>
>     </dependency>
>
>     <dependency>
>       <groupId>org.eclipse.jetty</groupId>
>       <artifactId>*jetty-servlet*</artifactId>
>     </dependency>
>
> Pretty much all of the missing class definition errors came up while
> building HttpServer.scala, and went away after the above dependencies were
> included.
>
> My guess is official build for spark 1.2 is working already. My question
> is what is wrong with my environment or setup, that requires me to add
> dependencies to pom.xml in this manner, to get this build to succeed.
>
> Also, i am not sure if this build would work at runtime for us, i am still
> testing this out.
>
>
> Thanks,
> pala
>

Mime
View raw message