kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Federico Giraud <giraud.feder...@gmail.com>
Subject High delay during controlled shutdown and acks=-1
Date Mon, 02 Nov 2015 13:52:54 GMT
Hi,

I have few java async producers sending data to a 4-node Kafka cluster
version 0.8.2, containing few thousand topics, all with replication factor
2. When i use acks=1 and trigger a controlled shutdown + restart on one
broker, the producers will send data to the new leader, reporting a very
low additional delay during the transition (as expected). However if i use
acks=-1, the producers will report a ~15 seconds delay between the send and
the future.get. Is this behavior expected? Is there a way to make it
faster? Or maybe it is a problem with my configuration?

Broker configs:
broker.id=0
log.dirs=/var/kafka/md1/kafka-logs
zookeeper.connect=10.40.27.107,10.40.27.108,10.40.27.109
auto.create.topics.enable=true
default.replication.factor=2
delete.topic.enable=true
log.retention.hours=24
num.io.threads=5

Producer configs:
acks = -1
retries = 3
timeout.ms = 3000
batch.size = 1048576
linger.ms= 100
metadata.fetch.timeout.ms = 5000
metadata.max.age.ms = 60000

I tried different configurations, but i wasn't able to reduce the delay
during broker restart. The logs in the broker indicate that the controlled
shutdown was successful.

Thank you

regards,
Federico

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message