kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 陈小军 <chenxiao...@nhn.com>
Subject Re: kafka server crash when I insert the data
Date Fri, 27 Dec 2013 06:27:54 GMT
I use the librdkafka, a c libray of Kafka. 
https://github.com/edenhill/librdkafka
maybe there is1k logs/second.

Best Regards
Jerry
-----Original Message-----
From: "Guozhang Wang"&lt;wangguoz@gmail.com&gt; 
To: "users@kafka.apache.org"&lt;users@kafka.apache.org&gt;; "陈小军"&lt;chenxiaojun@nhn.com&gt;;

Cc: 
Sent: 2013-12-27 (星期五) 15:15:19
Subject: Re: kafka server crash when I insert the data
 
How do you insert the data into Kafka brokers?
 Guozhang 

On Thu, Dec 26, 2013 at 6:52 PM, 陈小军 &lt;chenxiaojun@nhn.com&gt; wrote:

I have 4 kafka brokers, and create three topics, each topic has four partions and 3 repilca.
 I use the kafka-trunk code. When I insert the data to the kafka, suddenly 3 of brokers dead,
and I check the logs, the error is following


[2013-12-27 11:39:13,987] INFO Reconnect due to socket error: null (kafka.consumer.SimpleConsumer)

gs,1] -&amp;gt; PartitionFetchInfo(3158155,200000000),[nelo2-crash-logs,3] -&amp;gt;
PartitionFetchInfo(15315,200000000) (kafka.server.ReplicaFetcherThread)

java.net.ConnectException: źε2;9Hat sun.nio.ch.Net.connect0(Native Method)



        at sun.nio.ch.Net.connect(Net.java:465)

        at sun.nio.ch.Net.connect(Net.java:457)

        at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:666)

        at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)

        at kafka.consumer.SimpleConsumer.connect(SimpleConsumer.scala:44)

        at kafka.consumer.SimpleConsumer.reconnect(SimpleConsumer.scala:57)

        at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:79)

        at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:71)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:110)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110)

        at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:109)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109)

        at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)

        at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:108)

                                                                                         
                                                                                         
                                       5149,2-9      88%


        at kafka.consumer.SimpleConsumer.reconnect(SimpleConsumer.scala:57)

        at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:79)

        at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:71)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:110)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110)

           kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:109)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109)

        at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)

        at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:108)

        at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:96)

        at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:88)

        at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)

[2013-12-27 11:39:13,986] WARN [ReplicaFetcherThread-1-3], Error in fetch Name: FetchRequest;
Version: 0; CorrelationId: 20401; ClientId: ReplicaFetcherThread-1-3; ReplicaId: 2; MaxWait:
500 ms; MinBytes: 1 bytes; RequestInfo: [nelo2-nor


mal-logs,1] -&amp;gt; PartitionFetchInfo(3158155,200000000),[nelo2-crash-logs,3] -&amp;gt;
PartitionFetchInfo(15315,200000000) (kafka.server.ReplicaFetcherThread)

java.net.ConnectException: źε9;9Hat sun.nio.ch.Net.connect0(Native Method)



        at sun.nio.ch.Net.connect(Net.java:465)

        at sun.nio.ch.Net.connect(Net.java:457)

        at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:666)

        at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)

        at kafka.consumer.SimpleConsumer.connect(SimpleConsumer.scala:44)

        at kafka.consumer.SimpleConsumer.reconnect(SimpleConsumer.scala:57)

        at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:79)

        at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:71)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:110)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110)

        at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:109)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109)

        at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109)

        at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)

        at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:108)

        at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:96)

        at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:88)

        at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)

[2013-12-27 11:39:13,986] INFO Reconnect due to socket error: null (kafka.consumer.SimpleConsumer)

[2013-12-27 11:39:13,987] INFO Reconnect due to socket error: null (kafka.consumer.SimpleConsumer)



Can someone alway meet this problem? and I don't why this problem is occur?





Best Regards

Jerry



-- 
-- Guozhang 



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message