spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Can not import KafkaProducer in spark streaming job
Date Sun, 01 May 2016 13:30:35 GMT
According to
examples/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala
:

import org.apache.kafka.clients.producer.{KafkaProducer, ProducerConfig,
ProducerRecord}

Can you give the command line you used to submit the job ?

Probably classpath issue.

On Sun, May 1, 2016 at 5:11 AM, fanooos <dev.fanooos@gmail.com> wrote:

> I have a very strange problem.
>
> I wrote a spark streaming job that monitor an HDFS directory, read the
> newly
> added files, and send the contents to Kafka.
>
> The job is written in python and you can got the code from this link
>
> http://pastebin.com/mpKkMkph
>
> When submitting the job I got that error
>
> *ImportError: cannot import name KafkaProducer*
>
> As you see, the error is very simple but the problem is that I could import
> the KafkaProducer from both python and pyspark shells without any problem.
>
> I tried to reboot the machine but the situation remain the same.
>
> What do you think the problem is?
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Can-not-import-KafkaProducer-in-spark-streaming-job-tp26857.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message