spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix C <>
Subject Re: Spark streaming with Kafka- couldnt find KafkaUtils
Date Tue, 07 Apr 2015 09:06:37 GMT
Or you could build an uber jar ( you could google that )

--- Original Message ---

From: "Akhil Das" <>
Sent: April 4, 2015 11:52 PM
To: "Priya Ch" <>
Cc:, "dev" <>
Subject: Re: Spark streaming with Kafka- couldnt find KafkaUtils

How are you submitting the application? Use a standard build tool like
maven or sbt to build your project, it will download all the dependency
jars, when you submit your application (if you are using spark-submit, then
use --jars option to add those jars which are causing
classNotFoundException). If you are running as a standalone application
without using spark-submit, then while creating the SparkContext, use
sc.addJar() to add those dependency jars.

For Kafka streaming, when you use sbt, these will be jars that are required:


Best Regards

On Sun, Apr 5, 2015 at 12:00 PM, Priya Ch <>

> Hi All,
>   I configured Kafka  cluster on a  single node and I have streaming
> application which reads data from kafka topic using KafkaUtils. When I
> execute the code in local mode from the IDE, the application runs fine.
> But when I submit the same to spark cluster in standalone mode, I end up
> with the following exception:
> java.lang.ClassNotFoundException:
> org/apache/spark/streaming/kafka/KafkaUtils.
> I am using spark-1.2.1 version. when i checked the source files of
> streaming, the source files related to kafka are missing. Are these not
> included in spark-1.3.0 and spark-1.2.1 versions ?
> Have to manually include these ??
> Regards,
> Padma Ch

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message