kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joong Lee <jo...@me.com>
Subject Re: Data replication and zero data loss
Date Fri, 01 May 2015 11:43:06 GMT
It is based on our understanding from reading the documents. 

We aren't concerned of data duplication as that is going to be handled by elasticsearch. 

> On May 1, 2015, at 12:15 AM, Daniel Compton <daniel.compton.lists@gmail.com> wrote:
> 
> When we evaluated MirrorMaker last year we didn't find any risk of data
> loss, only duplicate messages in the case of a network partition.
> 
> Did you discover data loss in your tests, or were you just looking at the
> docs?
> On Fri, 1 May 2015 at 4:31 pm Jiangjie Qin <jqin@linkedin.com.invalid>
> wrote:
> 
>> Which mirror maker version did you look at? The MirrorMaker in trunk
>> should not have data loss if you just use the default setting.
>> 
>>> On 4/30/15, 7:53 PM, "Joong Lee" <joong@me.com> wrote:
>>> 
>>> Hi,
>>> We are exploring Kafka to keep two data centers (primary and DR) running
>>> hosts of elastic search nodes in sync. One key requirement is that we
>>> can't lose any data. We POC'd use of MirrorMaker and felt it may not meet
>>> out data loss requirement.
>>> 
>>> I would like ask the community if we should look for another solution or
>>> would Kafka be the right solution considering zero data loss requirement.
>>> 
>>> Thanks
>> 
>> 

Mime
View raw message