kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Re: is there a way to make sure two consumers receive the same message from the broker?
Date Mon, 07 Nov 2016 16:48:17 GMT
Hi AmitHossein,

I still don't see how that guarantees consistency at any given time. other
words how do I know at time X the data in Cassandra and ES are the same.

Thanks


On Mon, Nov 7, 2016 at 3:26 AM, AmirHossein Roozbahany <divergnt@outlook.com
> wrote:

> Hi
>
> Can you use elasticsearch _version field as cassandra's
> writetime?(_version is strictly increasing, cassandra uses writetime for
> applying LWW, so last write in elasticsearch will always win)
>
> It needs no transaction and makes databases convergent.
>
>
> ________________________________
> From: kant kodali <kanth909@gmail.com>
> Sent: Monday, November 7, 2016 3:08 AM
> To: users@kafka.apache.org
> Subject: Re: is there a way to make sure two consumers receive the same
> message from the broker?
>
> Hi Hans,
>
> The two storages we use are Cassandra and Elastic search and they are on
> the same datacenter for now.
> The Programming Language we use is Java and OS would be Ubuntu or CentOS.
> We get messages in JSON format so we insert into Elastic Search directly
> and for Cassandra we transform JSON message into appropriate model so we
> could insert into a Cassandra table.
> The rate we currently get is about 100K/sec which is awesome but I am
> pretty sure this will go down once when we implement 2PC or transactional
> writes.
>
> Thanks,
> kant
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message