spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From durin <m...@simon-schaefer.net>
Subject Re: sbt package failed: wrong libraryDependencies for spark-streaming?
Date Fri, 01 Aug 2014 02:33:08 GMT
Hi Tathagata,

I was using the "raw" tag in the web-editor. Seems like this doesn't make it
into the mail.
Here's the message again, this time without those tags:



I've added the following to my spark-env.sh:

SPARK_CLASSPATH="/disk.b/spark-master-2014-07-28/external/twitter/target/spark-streaming-twitter_2.10-1.1.0-SNAPSHOT.jar"

I can now execute

import org.apache.spark.streaming.twitter._
import org.apache.spark.streaming.StreamingContext._

without an error in the shell. However, I will get an error when doing this:

scala> val ssc = new StreamingContext(sc, Seconds(1))
ssc: org.apache.spark.streaming.StreamingContext =
org.apache.spark.streaming.StreamingContext@6e78177b

scala> val tweets = TwitterUtils.createStream(ssc, "twitter.txt")
error: bad symbolic reference. A signature in TwitterUtils.class refers to
term twitter4j
in package <root> which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling
TwitterUtils.class.


What am I missing? Do I have to import another jar? 



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/sbt-package-failed-wrong-libraryDependencies-for-spark-streaming-tp11103p11116.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message