spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shivaram Venkataraman <shivaram.venkatara...@gmail.com>
Subject Re: N00b alert - spark-ec2 script failure
Date Fri, 13 Sep 2013 18:35:38 GMT
Right - you will only encounter this for the day 2 exercises (MLI,
BlinkDB). I pushed out a fix for this [1] that should be effective if
you launch another cluster.

Thanks
Shivaram

[1] https://github.com/mesos/spark-ec2/commit/4465aab125c4ee486da00136b7f107162c30f9f8

On Fri, Sep 13, 2013 at 11:17 AM, Venkat Krishnamurthy
<venkat@yarcdata.com> wrote:
> Just an update - I actually managed to work through the Day 1 tutorial
> with zero hiccups so far (up to the twitter exercise) so this didn't
> really didn't prevent me from working.
>
> Will update if I face any issues on day 2 exercises.
>
> I'm still picking my jaw up off the floor. Looking forward to the meetup.
>
> Venkat
>
> On 9/13/13 11:11 AM, "Shivaram Venkataraman"
> <shivaram.venkataraman@gmail.com> wrote:
>
>>Actually there is a problem with the AMP Camp 3 scripts now that the
>>package names have changed. We pull from master of MLI, BlinkDB that
>>depend on org.apache.spark but the package published on the AMI is
>>with old package names.  I will push out script changes that will sync
>>MLI and BlinkDB to the version that we used for the Camp and ping this
>>thread soon.
>>
>>Thanks
>>Shivaram
>>
>>On Fri, Sep 13, 2013 at 8:05 AM, Jey Kottalam <jey@cs.berkeley.edu> wrote:
>>> Hi Venkat,
>>>
>>> The 0.8.0 branch has not yet been released (and furthermore, the
>>> 0.8.0-SNAPSHOT version refers to a prerelease build). You can install
>>> Spark 0.8.0-SNAPSHOT to your local repository by checking out the
>>> latest source code from http://github.com/mesos/spark and running:
>>>
>>>     sbt/sbt publish-local
>>>
>>> If you are using Hadoop, specify the Hadoop version required as follows:
>>>
>>>    SPARK_HADOOP_VERSION=1.0.4 sbt/sbt publish-local
>>>
>>> Hope that helps,
>>> -Jey
>>>
>>> On Thu, Sep 12, 2013 at 10:09 PM, Venkat Krishnamurthy
>>> <venkat@yarcdata.com> wrote:
>>>> All
>>>>
>>>> Apologies if this post is misdirected. I'm trying to get started with
>>>>Spark,
>>>> and using the AMPLab hands-on as a guide ­ The set up went as far as
>>>> creating and spinning up the AMI's, but part way through is this
>>>>dependency
>>>> resolution error from sbt:
>>>>
>>>> Seems simple enough, but the failed dependency is spark-core҆ which I
>>>>assume
>>>> is An Important Thing.
>>>>
>>>> I'm hoping this is embarrassingly obvious to find and fix (or ignore)
>>>>- any
>>>> pointers?
>>>>
>>>> -----
>>>>
>>>> [warn] module not found:
>>>>org.apache.spark#spark-core_2.9.3;0.8.0-SNAPSHOT
>>>> [warn] ==== local: tried
>>>> [warn]
>>>>
>>>>/root/.ivy2/local/org.apache.spark/spark-core_2.9.3/0.8.0-SNAPSHOT/ivys/
>>>>ivy.xml
>>>> [warn] ==== Typesafe Repository: tried
>>>> [warn]
>>>>
>>>>http://repo.typesafe.com/typesafe/releases/org/apache/spark/spark-core_2
>>>>.9.3/0.8.0-SNAPSHOT/spark-core_2.9.3-0.8.0-SNAPSHOT.pom
>>>> [warn] ==== JBoss Repository: tried
>>>> [warn]
>>>>
>>>>http://repository.jboss.org/nexus/content/repositories/releases/org/apac
>>>>he/spark/spark-core_2.9.3/0.8.0-SNAPSHOT/spark-core_2.9.3-0.8.0-SNAPSHOT
>>>>.pom
>>>> [warn] ==== Spray Repository: tried
>>>> [warn]
>>>>
>>>>http://repo.spray.cc/org/apache/spark/spark-core_2.9.3/0.8.0-SNAPSHOT/sp
>>>>ark-core_2.9.3-0.8.0-SNAPSHOT.pom
>>>> [warn] ==== Cloudera Repository: tried
>>>> [warn]
>>>>
>>>>https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/sp
>>>>ark/spark-core_2.9.3/0.8.0-SNAPSHOT/spark-core_2.9.3-0.8.0-SNAPSHOT.pom
>>>> [warn] ==== Local Maven: tried
>>>> [warn]
>>>>
>>>>file:/root/.m2/repository/org/apache/spark/spark-core_2.9.3/0.8.0-SNAPSH
>>>>OT/spark-core_2.9.3-0.8.0-SNAPSHOT.pom
>>>> [warn] ==== public: tried
>>>> [warn]
>>>>
>>>>http://repo1.maven.org/maven2/org/apache/spark/spark-core_2.9.3/0.8.0-SN
>>>>APSHOT/spark-core_2.9.3-0.8.0-SNAPSHOT.pom
>>>> [info] Resolving org.apache.spark#spark-repl_2.9.3;0.8.0-SNAPSHOT ...
>>>> [warn] module not found:
>>>>org.apache.spark#spark-repl_2.9.3;0.8.0-SNAPSHOT
>>>> [warn] ==== local: tried
>>>> [warn]
>>>>
>>>>/root/.ivy2/local/org.apache.spark/spark-repl_2.9.3/0.8.0-SNAPSHOT/ivys/
>>>>ivy.xml
>>>> [warn] ==== Typesafe Repository: tried
>>>> [warn]
>>>>
>>>>http://repo.typesafe.com/typesafe/releases/org/apache/spark/spark-repl_2
>>>>.9.3/0.8.0-SNAPSHOT/spark-repl_2.9.3-0.8.0-SNAPSHOT.pom
>>>> [warn] ==== JBoss Repository: tried
>>>> [warn]
>>>>
>>>>http://repository.jboss.org/nexus/content/repositories/releases/org/apac
>>>>he/spark/spark-repl_2.9.3/0.8.0-SNAPSHOT/spark-repl_2.9.3-0.8.0-SNAPSHOT
>>>>.pom
>>>> [warn] ==== Spray Repository: tried
>>>> [warn]
>>>>
>>>>http://repo.spray.cc/org/apache/spark/spark-repl_2.9.3/0.8.0-SNAPSHOT/sp
>>>>ark-repl_2.9.3-0.8.0-SNAPSHOT.pom
>>>> [warn] ==== Cloudera Repository: tried
>>>> [warn]
>>>>
>>>>https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/sp
>>>>ark/spark-repl_2.9.3/0.8.0-SNAPSHOT/spark-repl_2.9.3-0.8.0-SNAPSHOT.pom
>>>> [warn] ==== Local Maven: tried
>>>> [warn]
>>>>
>>>>file:/root/.m2/repository/org/apache/spark/spark-repl_2.9.3/0.8.0-SNAPSH
>>>>OT/spark-repl_2.9.3-0.8.0-SNAPSHOT.pom
>>>> [warn] ==== public: tried
>>>> [warn]
>>>>
>>>>http://repo1.maven.org/maven2/org/apache/spark/spark-repl_2.9.3/0.8.0-SN
>>>>APSHOT/spark-repl_2.9.3-0.8.0-SNAPSHOT.pom
>>>> [info] Resolving org.scala-tools.testing#test-interface;0.5 ...
>>>> [warn] ::::::::::::::::::::::::::::::::::::::::::::::
>>>> [warn] ::          UNRESOLVED DEPENDENCIES         ::
>>>> [warn] ::::::::::::::::::::::::::::::::::::::::::::::
>>>> [warn] :: org.apache.spark#spark-core_2.9.3;0.8.0-SNAPSHOT: not found
>>>> [warn] :: org.apache.spark#spark-repl_2.9.3;0.8.0-SNAPSHOT: not found
>>>> [warn] ::::::::::::::::::::::::::::::::::::::::::::::
>>>> sbt.ResolveException: unresolved dependency:
>>>> org.apache.spark#spark-core_2.9.3;0.8.0-SNAPSHOT: not found
>>>>
>

Mime
View raw message