kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Stein <joe.st...@stealth.ly>
Subject Re: how to achieve availability with no data loss when replica = 1?
Date Wed, 10 Dec 2014 16:37:14 GMT
If you want no data loss then you need to set ack = -1
Copied from https://kafka.apache.org/documentation.html#producerconfigs ==
-1, which means that the producer gets an acknowledgement after all in-sync
replicas have received the data. This option provides the best durability,
we guarantee that no messages will be lost as long as at least one in sync
replica remains.

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/

On Wed, Dec 10, 2014 at 11:27 AM, Helin Xiang <xkeegan@gmail.com> wrote:

> Hi,
>
> in some topics of our system, the data volumn is so huge that we think
> doing extra replica is a waste of disk and network resource( plus the data
> is not so important).
>
> firstly, we use 1 replica + ack=0, found when 1 broker is down, the data
> would loss 1/n.
> then we tried 1 replica + ack=1, and found after 3 tries, the data is still
> lost. and when we set the try number large enough, no more data can be
> produced to Kafka.
>
> If I did not misunderstand, In 0.7, when 1 broker is down, both producing
> and consuming are available with no data loss. I can see the reason why
> kafka 0.8 is designed to be different like 0.7.  but is there a way to let
> producer of 0.8 act like the behavior in 0.7?  We don't care which part of
> data should go to the specific partition, as long as the data goes into the
> kafka with no loss.
>
>
> Thanks
>
>
>
>
> --
>
>
> *Best RegardsXiang Helin*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message