spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From durin <>
Subject sbt package failed: wrong libraryDependencies for spark-streaming?
Date Fri, 01 Aug 2014 00:48:07 GMT
As suggested  here

, I want to create a minimal project using sbt to be able to use
org.apache.spark.streaming.twitter in the shell. My Spark version is the
latest Master branch compiled with maven3.

I was following  this
tutorial to create a small standalone application.

My simple.sbt file looks like this:

However, I will get this error when using sbt package:

I don't quite understand the whole standalone thing yet. sbt is installed on
a different machine than my Spark Cluster, but the tutorial never mentions
any variables that would have to be set. So I did assume sbt didn't actually
need a Spark installation on the machine.
Also, I could imagine that 1.0.1 isn't the correct version when I compiled
it myself.

Can someone help me out? In the end, all I want for now is to be able to use
twitterUtils in the spark-shell.


View this message in context:
Sent from the Apache Spark User List mailing list archive at

View raw message