spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lannyripple <lanny.rip...@gmail.com>
Subject Re: spark streaming, kafka, SPARK_CLASSPATH
Date Thu, 19 Jun 2014 19:00:26 GMT
Gino,

I can confirm that your solution of assembling with spark-streaming-kafka
but excluding spark-core and spark-streaming has me working with basic
spark-submit.  As mentioned you must specify the assembly jar in the
SparkConfig as well as to spark-submit.

When I see the error you are now experiencing I just restart my cluster
(sbin/stop-all.sh; sleep 6; sbin/start-all.sh).  My thought is a resource
leak somewhere but I haven't tried to chase it down since restarting is nice
and quick.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-streaming-kafka-SPARK-CLASSPATH-tp7356p7941.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message