spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <c...@koeninger.org>
Subject Re: spark streaming program failed on Spark 1.4.1
Date Mon, 03 Aug 2015 13:46:53 GMT
Just to be clear, did you rebuild your job against spark 1.4.1 as well as
upgrading the cluster?

On Mon, Aug 3, 2015 at 8:36 AM, Netwaver <wanglong_428@163.com> wrote:

> Hi All,
>          I have a spark streaming + kafka program written by Scala, it
> works well on Spark 1.3.1, but after I migrate my Spark cluster to 1.4.1
> and rerun this program, I meet below exception:
>             ERROR scheduler.ReceiverTracker: Deregistered receiver for
> stream 0: Error starting receiver 0 - java.lang.NoClassDefFoundError:
> org/I0Itec/zkclient/serialize/ZkSerializer
>         at kafka.consumer.Consumer$.create(ConsumerConnector.scala:94)
>         at
> org.apache.spark.streaming.kafka.KafkaReceiver.onStart(KafkaInputDStream.scala:100)
>         at
> org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:125)
>         at
> org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:109)
>         at
> org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverLauncher$$anonfun$8.apply(ReceiverTracker.scala:308)
>         at
> org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverLauncher$$anonfun$8.apply(ReceiverTracker.scala:300)
>         at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1767)
>         at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1767)
>         at
> org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>         at org.apache.spark.scheduler.Task.run(Task.scala:70)
>         at
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.ClassNotFoundException:
> org.I0Itec.zkclient.serialize.ZkSerializer
>         at java.net.URLClassLoader$1.run(Unknown Source)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(Unknown Source)
>         at java.lang.ClassLoader.loadClass(Unknown Source)
>         at java.lang.ClassLoader.loadClass(Unknown Source)
>         ... 14 more
>       I did some web searching, and tried to add zkclient-0.3 jar in the
> classpath, but still get same issue, Who can share your experience on
> solving this issue? thanks in advance.
>
>
>
>

Mime
View raw message