storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 王 纯超 <wangchunc...@outlook.com>
Subject Re: Re: Executor Change
Date Sat, 05 Aug 2017 09:25:40 GMT
Thanks, Bobby. Stateful bolt requires reliable processing guarantee, but in my case, it is
abandoned for my specific requirement. Thus I tried to save the state myself in the bolt's
close method, which did not work. The state was not saved. So how to store the state and when
is close method called?

________________________________
wangchunchao@outlook.com

From: Bobby Evans<mailto:evans@yahoo-inc.com>
Date: 2017-08-04 21:47
To: user@storm.apache.org<mailto:user@storm.apache.org>
CC: zhangxl_bds<mailto:zhangxl_bds@si-tech.com.cn>
Subject: Re: Executor Change
Storm is by default stateless.  If you have state to store it is up to you to store that state
in an external system, in a way that meets your topologies requirements.  At least one vs
at most once.  In newer versions of storm the concept of stateful bolts was added.

http://storm.apache.org/releases/1.0.3/State-checkpointing.html

is the documentation for that feature.  It still saves the state in an external system (that
you can pick) but also helps to fix some of the complexity associated with checkpointing and
restoring state in line with replay of failed messages.

- Bobby



On Friday, August 4, 2017, 12:45:17 AM CDT, ? ?? <wangchunchao@outlook.com> wrote:


Hi,

    It is observed that the executor of a task changes to a different worker on  a different
supervisor. And the computation, which is similar to wordcount,  stops and starts over. After
several tests, I find that this is caused by the lost of the task state during executor change.
So I wonder:

  1.  Why does the change happen?
  2.  When the change happen, does the executor use a new component(bolt) instance? How does
the instance come into being? By constructor or deserialization?
  3.  In this case, how to persist the task state?

________________________________
wangchunchao@outlook.com
Mime
View raw message