kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sachin Mittal <sjmit...@gmail.com>
Subject Difference between with and without state store cleanup streams startup
Date Fri, 31 Mar 2017 18:46:07 GMT
Hi,
There are two ways to re start a streams application
1. executing streams.cleanUp() before streams.start()
This cleans up the local state store.

2. Just by calling streams.start()

What are the differences between two.

As I understand in first case it will try to create local state store by
replaying the changelog topic.

So questions here are
1. Will it try to replay the whole log from earliest or from last committed
offset?
2. Will it read and fetch all the values from the topic for a given key or
only the last value for a key when creating a state store for that change
log topic?
3. What happens if time to create the state store is greater than
max.poll.interval.ms

4. If we don't delete the state store then what happens. Does it again try
to recreate the same store by reading entire change log topic? Or if it
determines somehow from the state store what the latest offset is and
updates the state store with values from that offset onwards.

5. In case of unclean shutdown say by kill -9 is it advisable to cleanup
local state store before restart or we can just stat it normally.

6. Finally can anyone point me to the code where it creates the state store
by reading from changelog topic.

Thanks
Sachin

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message