kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: Kafka Streams - StateRestoreListener called when new partitions assigned
Date Mon, 11 Nov 2019 22:43:12 GMT
Hello Javier,

When a rebalance happened and the new tasks (hence input partitions) are
assigned that need to be restored, the state of the instance would also
transit to REBALANCING, and would only be transit back to RUNNING after all
tasks have been completed restoring and all are being processed normally.

Looking at the docs, I think we can add a specific paragraph on web docs
"developer guide" on exactly when the state transition would happen to
clarify those things, could you create a JIRA ticket for improving on our
docs so that we would not drop this on the floor?

Guozhang

On Tue, Nov 12, 2019 at 4:16 AM Javier Holguera <javier.holguera@gmail.com>
wrote:

> Hi,
>
> I understand that the state store listener that can be set using
> KafkaStreams.setGlobalStateRestoreListener will be invoked when the streams
> app starts if it doesn't find the state locally (e.g., running on a
> ephemeral docker container).
>
> However, I wonder if the process happens as well if there is a rebalance
> and the streams app gets assigned new partitions for which it doesn't have
> state yet. Does the app go into the rebalancing state and stays there until
> the state for those new partitions have been restored? Or does it go back
> to running even if it hasn't happened yet?
>
> Thanks.
>
> Regards,
> Javier.
>


-- 
-- Guozhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message