storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun Mahadevan <ar...@apache.org>
Subject Re: Rebalancing Stateful bolts in storm 1.0.2
Date Mon, 20 Feb 2017 04:27:10 GMT
This is expected with in-memory state, which stores the state in a local hash map and is not
intended for any real use cases. And I don’t think there is any value in serializing the
in-memory state during rebalance. How would you resurrect the state if the task gets reassigned
to a different host?

 

Better use the redis state implementation or write a state implementation that uses distributed
memory like on top of memcached.

 

Arun

 

From: anshu shukla <anshushukla0@gmail.com>
Reply-To: "user@storm.apache.org" <user@storm.apache.org>
Date: Monday, February 20, 2017 at 8:50 AM
To: "user@storm.apache.org" <user@storm.apache.org>
Subject: Rebalancing Stateful bolts in storm 1.0.2

 

Hey,

I was running in-memory stateful bolt and while doing rebalance the state for the tasks get
lost. 

Can anyone suggest some work around to retain the state while rebalancing without using stores
like Redis (like on rebalancing I should serialize the state along with the task while regrouping
task to other threads)? 

please suggest some general idea about the possibilities.

 

-- 

Thanks & Regards,
Anshu Shukla


Mime
View raw message