spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bharath Bhushan <manku.ti...@outlook.com>
Subject Re: java.lang.ClassNotFoundException - spark on mesos
Date Wed, 02 Apr 2014 09:30:06 GMT
I tried several things in order to get 1.0.0 git tree to work with 
mesos. All my efforts failed. I could run spark 0.9.0 on mesos but not 
spark 1.0.0. Please suggest any other things I can try.

1. Change project/SparkBuild.scala to use mesos 0.17.0 and then 
make_distribution.sh.

2. Try building with maven. Threw the following error:
[INFO] Spark Project Parent POM .......................... SUCCESS 
[7:55.796s]
[INFO] Spark Project Core ................................ FAILURE [7.209s]
...
[ERROR] Plugin org.apache.maven.plugins:maven-compiler-plugin:3.1 or one 
of its
dependencies could not be resolved: Failed to read artifact descriptor 
for org.a
pache.maven.plugins:maven-compiler-plugin:jar:3.1: Could not find 
artifact org.a
pache:apache:pom:13 -> [Help 1]

3. Change pom.xml to use mesos 0.17.0 and protobuf 2.5.0 and then 
make_distribution.sh

4. Change pom.xml to use mesos 0.17.0 and leave protobuf at 2.4.1 and 
then make_distribution.sh

5. Change pom.xml to use mesos 0.17.0 and remove the protobuf line and 
then make_distribution.sh

Question: Is the pom.xml modification picked up when building with sbt 
and maven or only when building with maven?

Thanks

On 01/04/14 11:04 am, Bharath Bhushan wrote:
> Another problem I noticed is that the current 1.0.0 git tree still gives me the ClassNotFoundException.
I see that the SPARK-1052 is already fixed there. I then modified the pom.xml for mesos and
protobuf and that still gave the ClassNotFoundException. I also tried modifying pom.xml only
for mesos and that fails too. So I have no way of running the 1.0.0 git tree spark on mesos
yet.
>
> Thanks.
>
> On 01-Apr-2014, at 3:28 am, deric <barton.tomas@gmail.com> wrote:
>
>> Which repository do you use?
>>
>> The issue should be fixed in 0.9.1 and 1.0.0
>>
>> https://spark-project.atlassian.net/browse/SPARK-1052
>> <https://spark-project.atlassian.net/browse/SPARK-1052>
>>
>> There's an old repository
>>
>> https://github.com/apache/incubator-spark
>>
>> and as Spark become one of top level projects, it was moved to new repo:
>>
>> https://github.com/apache/spark
>>
>> The 0.9.1 version hasn't been released yet, so you should get it from the
>> new git repo.
>>
>>
>>
>> --
>> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/java-lang-ClassNotFoundException-spark-on-mesos-tp3510p3551.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.


Mime
View raw message