kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Bejeck <b...@confluent.io>
Subject Re: Doubts in KStreams
Date Wed, 21 Feb 2018 15:18:27 GMT
Hi Pravin,

1.  Fault tolerance means that state stores are backed by topics,
changelogs, storing the contents of the state store.  For example, in a
worst case scenario, your machine crashed destroying all your local state,
on starting your Kafka Streams application back up the state stores would
recover the data (up to the last committed offset) from their backing
changelog topics.

2. If you have a multiple instance Kafka Streams application (A and B) and
instance A dies, a rebalance occurs, and the tasks from A get assigned to
B.  If instance A had any local state then the new tasks on B use the
changelog topics to fill the state stores for the new task, so the local
state store is recovered up to the last committed offset of the state store
on A before it crashed.

HTH

-Bill

On Wed, Feb 21, 2018 at 5:40 AM, pravin kumar <pk007790@gmail.com> wrote:

> I have studied KafkaStreams, but not clearly understood
>
> 1.Can someone explain about Fault tolerence.
> 2.I have topicA and topicB with 4 partitions, so it created fourTasks, I
> have created it in singleJVM.But i need to knw how it works in multiple JVM
> and if one jvm goes down,how it another jvm takes the responsibility and
> how the localStateStore is recreated in the JVM which takes responsibility.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message