kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Becket Qin <becket....@gmail.com>
Subject Re: High delay during controlled shutdown and acks=-1
Date Mon, 02 Nov 2015 18:40:48 GMT
Hi Federico,

What is your replica.lag.time.max.ms?

When acks=-1, the ProducerResponse won't return until all the broker in ISR
get the message. During controlled shutdown, the shutting down broker is
doing a lot of leader migration and could slow down on fetching data. The
broker won't be kicked out of ISR until at least replica.lag.time.max.ms.
Reducing the configuration will let the shutting down broker to be kicked
out of ISR quicker if it cannot catch up. But if you set it too small,
there could be more ISR expansion/shrinking.

That said, currently controlled shutdown is not very efficient. We might
improve it and hopefully later on it won't slow down the replication on the
shutting down broker.

Thanks,

Jiangjie (Becket) Qin

On Mon, Nov 2, 2015 at 5:52 AM, Federico Giraud <giraud.federico@gmail.com>
wrote:

> Hi,
>
> I have few java async producers sending data to a 4-node Kafka cluster
> version 0.8.2, containing few thousand topics, all with replication factor
> 2. When i use acks=1 and trigger a controlled shutdown + restart on one
> broker, the producers will send data to the new leader, reporting a very
> low additional delay during the transition (as expected). However if i use
> acks=-1, the producers will report a ~15 seconds delay between the send and
> the future.get. Is this behavior expected? Is there a way to make it
> faster? Or maybe it is a problem with my configuration?
>
> Broker configs:
> broker.id=0
> log.dirs=/var/kafka/md1/kafka-logs
> zookeeper.connect=10.40.27.107,10.40.27.108,10.40.27.109
> auto.create.topics.enable=true
> default.replication.factor=2
> delete.topic.enable=true
> log.retention.hours=24
> num.io.threads=5
>
> Producer configs:
> acks = -1
> retries = 3
> timeout.ms = 3000
> batch.size = 1048576
> linger.ms= 100
> metadata.fetch.timeout.ms = 5000
> metadata.max.age.ms = 60000
>
> I tried different configurations, but i wasn't able to reduce the delay
> during broker restart. The logs in the broker indicate that the controlled
> shutdown was successful.
>
> Thank you
>
> regards,
> Federico
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message