kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damian Guy <damian....@gmail.com>
Subject Re: Kafka Streams - "state store may have migrated to another instance"
Date Wed, 07 Mar 2018 15:20:12 GMT
If you have multiple streams instances then the store might only be
available on one of the instances. Using `KafkaStreams.store(..)` will only
locate stores that are currently accessible by that instance. If you need
to be able to locate stores on other instances, then you should probably
have a read of:

There is also a convenient 3rd party lib that can help you with this:

On Wed, 7 Mar 2018 at 14:07 detharon <detharon@gmail.com> wrote:

> Hello, I'm experiencing issues accessing the state stores outside the Kafka
> stream. My application queries the state store every n seconds using the
> .all() method to retrieve all key value pairs. I know that the state store
> might not be available, so I guard against the InvalidStateStoreException
> and in case it's caught, I simply invoke the .store method on my stream, in
> order to get a new store. The problem is, that for some reason the store
> never becomes available.
> Some facts:
> - During that time stream processing works correctly, and it successfully
> puts and gets data to and from the store.
> - Stream is in "running" state. I've started logging that because under
> normal circumstances this exception is being thrown when the stream is in
> "rebalancing" phase, but after a while it's gone, as expected, so I can
> distinguish between these two cases.
> - It only happens when I run my app in multiple instances.
> I've set the log level to debug, but I can't see anything suspicious in the
> logs, but maybe there's something particular I should pay attention to?
> I access the store from inside an Akka actor, to which I pass the reference
> to KafkaStreams object, if that matters.
> I ran out of ideas what might have caused that behavior, so any help will
> be greatly appreciated.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message