kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bala <kbkre...@yahoo.com.INVALID>
Subject Kafka consumer error in storm topology
Date Mon, 01 Oct 2018 22:18:11 GMT
Not sure if this is a kafka issue,but I am seeing this on a kerberized cluster and seeing this
exception in kafka spout.
 java.nio.channels.ClosedChannelException: null
        at kafka.network.BlockingChannel.send(BlockingChannel.scala:122) ~[stormjar.jar:?]
        at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:101) [stormjar.jar:?]
        at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:99)
[stormjar.jar:?]
        at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:165)
[stormjar.jar:?]
        at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:86)
[stormjar.jar:?]
        at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:81) [stormjar.jar:?]
        at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:71) [stormjar.jar:?]
        at org.apache.storm.kafka.PartitionManager.<init>(PartitionManager.java:135)
[stormjar.jar:?]
        at org.apache.storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:110) [stormjar.jar:?]
        at org.apache.storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:71)
[stormjar.jar:?]
        at org.apache.storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:130) [stormjar.jar:?]
        at org.apache.storm.daemon.executor$fn__9567$fn__9582$fn__9613.invoke(executor.clj:647)
[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
        at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484) [storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
        at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]2018-10-01 20:17:31.955
o.a.s.util Thread-35-icdkafkaspout-executor[43 43] [ERROR] Async loop died! java.lang.RuntimeException:
java.nio.channels.ClosedChannelException
        at org.apache.storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:115) ~[stormjar.jar:?]
        at org.apache.storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:71)
~[stormjar.jar:?]
        at org.apache.storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:130) ~[stormjar.jar:?]
        at org.apache.storm.daemon.executor$fn__9567$fn__9582$fn__9613.invoke(executor.clj:647)
~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
        at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484) [storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
        at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: java.nio.channels.ClosedChannelException
        at kafka.network.BlockingChannel.send(BlockingChannel.scala:122) ~[stormjar.jar:?]
        at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:114) ~[stormjar.jar:?]
        at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:99)
~[stormjar.jar:?]
        at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:165)
~[stormjar.jar:?]
        at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:86)
~[stormjar.jar:?]
        at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:81) ~[stormjar.jar:?]
        at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:71) ~[stormjar.jar:?]
        at org.apache.storm.kafka.PartitionManager.<init>(PartitionManager.java:135)
~[stormjar.jar:?]
        at org.apache.storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:110) ~[stormjar.jar:?]       
... 6 more2018-10-01 20:17:31.956 o.a.s.d.executor Thread-35-icdkafkaspout-executor[43 43]
[ERROR] java.lang.RuntimeException: java.nio.channels.ClosedChannelException
        at org.apache.storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:115) ~[stormjar.jar:?]
        at org.apache.storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:71)
~[stormjar.jar:?]
        at org.apache.storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:130) ~[stormjar.jar:?]
        at org.apache.storm.daemon.executor$fn__9567$fn__9582$fn__9613.invoke(executor.clj:647)
~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
        at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484) [storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
        at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: java.nio.channels.ClosedChannelException
        at kafka.network.BlockingChannel.send(BlockingChannel.scala:122) ~[stormjar.jar:?]
        at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:114) ~[stormjar.jar:?]
        at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:99)
~[stormjar.jar:?]
        at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:165)
~[stormjar.jar:?]
        at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:86)
~[stormjar.jar:?]
        at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:81) ~[stormjar.jar:?]
        at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:71) ~[stormjar.jar:?]
        at org.apache.storm.kafka.PartitionManager.<init>(PartitionManager.java:135)
~[stormjar.jar:?]
        at org.apache.storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:110) ~[stormjar.jar:?]       
... 6 more2018-10-01 20:17:31.979 o.a.s.util Thread-35-icdkafkaspout-executor[43 43] [ERROR]
Halting process: ("Worker died")
java.lang.RuntimeException: ("Worker died")
        at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) [storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
        at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
        at org.apache.storm.daemon.worker$fn__10237$fn__10238.invoke(worker.clj:763)
[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
        at org.apache.storm.daemon.executor$mk_executor_data$fn__9453$fn__9454.invoke(executor.clj:276)
[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
        at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:494) [storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
        at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message