spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From durin <m...@simon-schaefer.net>
Subject sbt package failed: wrong libraryDependencies for spark-streaming?
Date Fri, 01 Aug 2014 00:48:07 GMT
As suggested  here
<http://apache-spark-user-list.1001560.n3.nabble.com/import-org-apache-spark-streaming-twitter-in-Shell-tp9665p9743.html>

, I want to create a minimal project using sbt to be able to use
org.apache.spark.streaming.twitter in the shell. My Spark version is the
latest Master branch compiled with maven3.

I was following  this
<http://spark.apache.org/docs/latest/quick-start.html#standalone-applications>  
tutorial to create a small standalone application.

My simple.sbt file looks like this:




However, I will get this error when using sbt package:




I don't quite understand the whole standalone thing yet. sbt is installed on
a different machine than my Spark Cluster, but the tutorial never mentions
any variables that would have to be set. So I did assume sbt didn't actually
need a Spark installation on the machine.
Also, I could imagine that 1.0.1 isn't the correct version when I compiled
it myself.


Can someone help me out? In the end, all I want for now is to be able to use
twitterUtils in the spark-shell.


Simon



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/sbt-package-failed-wrong-libraryDependencies-for-spark-streaming-tp11103.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message