kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hans Jespersen <h...@confluent.io>
Subject Re: about Exactly-once Semantics
Date Sun, 02 Jul 2017 07:03:21 GMT
When you write the msg results to MySQL, you include the offset of the message with the results.
This can be done in one atomic write transaction. Then if your application crashes, when it
starts back up, it should read the offset stored with the last message results in the database,
then seek() to that offset, and continue consuming with exactly once semantics.

This is how many of the exactly once Kafka Connect Sink Connectors work today.


> On Jul 1, 2017, at 11:28 PM, fuyou <fuyou001@gmail.com> wrote:
> I read the great blog about kafka Exactly-once Semantics
> <https://www.confluent.io/blog/exactly-once-semantics-are-possible-heres-how-apache-kafka-does-it/>
> .
> my question is when consumer receive msg,and process the msg  result save
> to db(eg.MySQL),how to keep Exactly-once Semantics.
> I think the kafka Exactly-once Semantics useful when consumer  process the
> msg result save to kafka.
> am i right ?
> thanks .
> -- 
>   =============================================
>  fuyou001
> Best Regards

View raw message