kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shushant Arora <shushantaror...@gmail.com>
Subject kafka producer error
Date Wed, 09 Sep 2015 02:07:37 GMT
Hi

I have a kafka cluster with 3 brokers. I have a topic with ~50 partitions
and replication factpr of 3.

When 2 brokers are down - I m getting below error in producer code

5/09/09 00:56:15 WARN network.Selector: Error in I/O with brokerIPXXXX(Ip
of broker which is down)
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.kafka.common.network.Selector.poll(Selector.java:238)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
at java.lang.Thread.run(Thread.java:745)

Producer config are :



value.serializer = class
org.apache.kafka.common.serialization.StringSerializer
key.serializer = class
org.apache.kafka.common.serialization.StringSerializer
block.on.buffer.full = true
retry.backoff.ms = 200
buffer.memory = 10485760
batch.size = 16384
metrics.sample.window.ms = 30000
metadata.max.age.ms = 300000
receive.buffer.bytes = 32768
timeout.ms = 30000
max.in.flight.requests.per.connection = 5
bootstrap.servers = [broker1:9092,broker2:9092,broker3:9092]
retries = 10
max.request.size = 1048576
send.buffer.bytes = 131072
acks = 1
reconnect.backoff.ms = 10
linger.ms = 0
metrics.num.samples = 2
metadata.fetch.timeout.ms = 60000


despite being acks as 1 - I m getting IOError - and its giving error for
broker which is down - but my topic has replication and when I see in kafka
console describe on topic - new leader of all partitions is broker which is
Up.

So why the IOException is coming?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message