kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kamal C <kamaltar...@gmail.com>
Subject Re: kafka producer error
Date Wed, 09 Sep 2015 06:12:23 GMT
Refer the mail list
http://qnalist.com/questions/6002514/new-producer-metadata-update-problem-on-2-node-cluster

https://issues.apache.org/jira/browse/KAFKA-1843

On Wed, Sep 9, 2015 at 7:37 AM, Shushant Arora <shushantarora09@gmail.com>
wrote:

> Hi
>
> I have a kafka cluster with 3 brokers. I have a topic with ~50 partitions
> and replication factpr of 3.
>
> When 2 brokers are down - I m getting below error in producer code
>
> 5/09/09 00:56:15 WARN network.Selector: Error in I/O with brokerIPXXXX(Ip
> of broker which is down)
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
> at org.apache.kafka.common.network.Selector.poll(Selector.java:238)
> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
> at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
> at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
> at java.lang.Thread.run(Thread.java:745)
>
> Producer config are :
>
>
>
> value.serializer = class
> org.apache.kafka.common.serialization.StringSerializer
> key.serializer = class
> org.apache.kafka.common.serialization.StringSerializer
> block.on.buffer.full = true
> retry.backoff.ms = 200
> buffer.memory = 10485760
> batch.size = 16384
> metrics.sample.window.ms = 30000
> metadata.max.age.ms = 300000
> receive.buffer.bytes = 32768
> timeout.ms = 30000
> max.in.flight.requests.per.connection = 5
> bootstrap.servers = [broker1:9092,broker2:9092,broker3:9092]
> retries = 10
> max.request.size = 1048576
> send.buffer.bytes = 131072
> acks = 1
> reconnect.backoff.ms = 10
> linger.ms = 0
> metrics.num.samples = 2
> metadata.fetch.timeout.ms = 60000
>
>
> despite being acks as 1 - I m getting IOError - and its giving error for
> broker which is down - but my topic has replication and when I see in kafka
> console describe on topic - new leader of all partitions is broker which is
> Up.
>
> So why the IOException is coming?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message