spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From manas Kar <Manas....@exactearth.com>
Subject Spark Streaming Example with CDH5
Date Wed, 18 Jun 2014 02:14:57 GMT
Hi Spark Gurus, 
 I am trying to compile a spark streaming example with CDH5 and having
problem compiling it. 
Has anyone created an example spark streaming using CDH5(preferably Spark
0.9.1) and would be kind enough to share the build.sbt(.scala) file?(or
point to their example on github). I know there is a streaming example  here
<https://github.com/apache/spark/tree/master/examples>   but I am looking
for something that runs with CDH5.


My build.scala files looks like given below.

 object Dependency {
    // Versions
    object V {
    val Akka = "2.3.0"
    val scala = "2.10.4" 
    val cloudera = "0.9.0-cdh5.0.0"
    }

val sparkCore      = "org.apache.spark" %% "spark-core"% V.cloudera
val sparkStreaming = "org.apache.spark" %% "spark-streaming" % V.cloudera

resolvers ++= Seq( "cloudera repo" at
"https://repository.cloudera.com/artifactory/cloudera-repos/",
                      "haddop repo" at
"https://repository.cloudera.com/content/repositories/releases/")

I have also attached the complete build.scala file for sake of completeness.
sbt dist gives the following error:
 object SecurityManager is not a member of package org.apache.spark
[error] import org.apache.spark.{SparkConf, SecurityManager}


build.scala
<http://apache-spark-user-list.1001560.n3.nabble.com/file/n7796/build.scala>  


Appreciate the great work the spark community is doing. It is by far the
best thing I have worked on.

..Manas



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Example-with-CDH5-tp7796.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message