spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pala M Muthaia <mchett...@rocketfuelinc.com>
Subject Re: Building spark 1.2 from source requires more dependencies
Date Thu, 26 Mar 2015 22:48:20 GMT
+spark-dev

Yes, the dependencies are there. I guess my question is how come the build
is succeeding in the mainline then, without adding these dependencies?

On Thu, Mar 26, 2015 at 3:44 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Looking at output from dependency:tree, servlet-api is brought in by the
> following:
>
> [INFO] +- org.apache.cassandra:cassandra-all:jar:1.2.6:compile
> [INFO] |  +- org.antlr:antlr:jar:3.2:compile
> [INFO] |  +- com.googlecode.json-simple:json-simple:jar:1.1:compile
> [INFO] |  +- org.yaml:snakeyaml:jar:1.6:compile
> [INFO] |  +- edu.stanford.ppl:snaptree:jar:0.1:compile
> [INFO] |  +- org.mindrot:jbcrypt:jar:0.3m:compile
> [INFO] |  +- org.apache.thrift:libthrift:jar:0.7.0:compile
> [INFO] |  |  \- javax.servlet:servlet-api:jar:2.5:compile
>
> FYI
>
> On Thu, Mar 26, 2015 at 3:36 PM, Pala M Muthaia <
> mchettiar@rocketfuelinc.com> wrote:
>
>> Hi,
>>
>> We are trying to build spark 1.2 from source (tip of the branch-1.2 at
>> the moment). I tried to build spark using the following command:
>>
>> mvn -U -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -Phive
>> -Phive-thriftserver -DskipTests clean package
>>
>> I encountered various missing class definition exceptions (e.g: class
>> javax.servlet.ServletException not found).
>>
>> I eventually got the build to succeed after adding the following set of
>> dependencies to the spark-core's pom.xml:
>>
>>     <dependency>
>>       <groupId>javax.servlet</groupId>
>>       <artifactId>*servlet-api*</artifactId>
>>       <version>3.0</version>
>>     </dependency>
>>
>>     <dependency>
>>       <groupId>org.eclipse.jetty</groupId>
>>       <artifactId>*jetty-io*</artifactId>
>>     </dependency>
>>
>>     <dependency>
>>       <groupId>org.eclipse.jetty</groupId>
>>       <artifactId>*jetty-http*</artifactId>
>>     </dependency>
>>
>>     <dependency>
>>       <groupId>org.eclipse.jetty</groupId>
>>       <artifactId>*jetty-servlet*</artifactId>
>>     </dependency>
>>
>> Pretty much all of the missing class definition errors came up while
>> building HttpServer.scala, and went away after the above dependencies were
>> included.
>>
>> My guess is official build for spark 1.2 is working already. My question
>> is what is wrong with my environment or setup, that requires me to add
>> dependencies to pom.xml in this manner, to get this build to succeed.
>>
>> Also, i am not sure if this build would work at runtime for us, i am
>> still testing this out.
>>
>>
>> Thanks,
>> pala
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message