spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: Spark Streaming Example with CDH5
Date Wed, 18 Jun 2014 07:02:25 GMT
There is nothing special about CDH5 Spark in this regard. CDH 5.0.x has
Spark 0.9.0, and the imminent next release will have 1.0.0 + upstream
patches.

You're simply accessing a class that was not present in 0.9.0, but is
present after that:

https://github.com/apache/spark/commits/master/core/src/main/scala/org/apache/spark/SecurityManager.scala




On Wed, Jun 18, 2014 at 3:14 AM, manas Kar <Manas.Kar@exactearth.com> wrote:

> Hi Spark Gurus,
>  I am trying to compile a spark streaming example with CDH5 and having
> problem compiling it.
> Has anyone created an example spark streaming using CDH5(preferably Spark
> 0.9.1) and would be kind enough to share the build.sbt(.scala) file?(or
> point to their example on github). I know there is a streaming example
>  here
> <https://github.com/apache/spark/tree/master/examples>   but I am looking
> for something that runs with CDH5.
>
>
> My build.scala files looks like given below.
>
>  object Dependency {
>     // Versions
>     object V {
>     val Akka = "2.3.0"
>     val scala = "2.10.4"
>     val cloudera = "0.9.0-cdh5.0.0"
>     }
>
> val sparkCore      = "org.apache.spark" %% "spark-core"% V.cloudera
> val sparkStreaming = "org.apache.spark" %% "spark-streaming" % V.cloudera
>
> resolvers ++= Seq( "cloudera repo" at
> "https://repository.cloudera.com/artifactory/cloudera-repos/",
>                       "haddop repo" at
> "https://repository.cloudera.com/content/repositories/releases/")
>
> I have also attached the complete build.scala file for sake of
> completeness.
> sbt dist gives the following error:
>  object SecurityManager is not a member of package org.apache.spark
> [error] import org.apache.spark.{SparkConf, SecurityManager}
>
>
> build.scala
> <
> http://apache-spark-user-list.1001560.n3.nabble.com/file/n7796/build.scala
> >
>
>
> Appreciate the great work the spark community is doing. It is by far the
> best thing I have worked on.
>
> ..Manas
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Example-with-CDH5-tp7796.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>

Mime
View raw message