spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <c...@koeninger.org>
Subject Re: Problem running JavaDirectKafkaWordCount
Date Mon, 14 Mar 2016 16:14:03 GMT
Sounds like the jar you built doesn't include the dependencies (in
this case, the spark-streaming-kafka subproject).  When you use
spark-submit to submit a job to spark, you need to either specify all
dependencies as additional --jars arguments (which is a pain), or
build an uber-jar containing all your dependencies, aside from
spark-core which is provided.

On Sat, Mar 12, 2016 at 10:01 AM, Martin Andreoni <martin@gta.ufrj.br> wrote:
> Hi, I'm starting  with spark and I'm having some issues.
>
> When I run the 'jar' example of JavaDirectKafkaWordCount it works perfectly.
> However, if I compile the code and when I submit it, I'm having the
> following error:
>
> ERROR ActorSystemImpl: Uncaught fatal error from thread
> [sparkDriver-akka.actor.default-dispatcher-2] shutting down ActorSystem
> [sparkDriver]
> java.lang.NoSuchMethodError:
> org.apache.spark.streaming.kafka.DirectKafkaInputDStream.id()I
>
>
>
> The program is running locally and the driver configuration is with 4g.
>
> Some help is always welcome,
>
> Thanks.
>
> --
>
>
> ________________________________
>
>
>
> Martin Andreoni

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message