kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stevo Slavić <ssla...@gmail.com>
Subject Kafka settings for (more) reliable/durable messaging
Date Tue, 07 Jul 2015 14:13:43 GMT
Hello Apache Kafka community,

Documentation for min.insync.replicas in
http://kafka.apache.org/documentation.html#brokerconfigs states:

"When used together, min.insync.replicas and request.required.acks allow
you to enforce greater durability guarantees. A typical scenario would be
to create a topic with a replication factor of 3, set min.insync.replicas
to 2, and produce with request.required.acks of -1. This will ensure that
the producer raises an exception if a majority of replicas do not receive a
write."

Correct me if wrong (doc reference?), I assume min.insync.replicas includes
lead, so with min.insync.replicas=2, lead and one more replica besides lead
will have to ACK writes.

In such setup, with minimalistic 3 brokers cluster, given that
- all 3 replicas are insync
- a batch of messages is written and ends up on lead and one replica ACKs
- another batch of messages ends up on lead and different replica ACKs

Is it possible that when lead crashes, while replicas didn't catch up,
(part of) one batch of messages could be lost (since one replica becomes a
new lead, and it's only serving all reads and requests, and replication is
one way)?

Kind regards,
Stevo Slavic.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message