storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahak Goel <mahakgoe...@gmail.com>
Subject Re: Spout sleep wait strategy
Date Tue, 15 Aug 2017 15:36:53 GMT
Text from post. 

2. Spout wait strategies: There's two situations in which a spout needs to wait. The first
is when the max spout pending limit is reached. The second is when nothing is emitted from
nextTuple. Previously, Storm would just have that spout sit in a busy loop in those cases.
What Storm does in those situations is now pluggable, and the default is now for the spout
to sleep for 1 ms. This will cause the spout to use dramatically less CPU when it hits those
cases, and it also obviates the need for spouts to do any sleeping in their implementation
to be "polite". The wait strategy can be configured with TOPOLOGY_SPOUT_WAIT_STRATEGY and
can be configured on a spout by spout basis. The interface to implement for a wait strategy
is backtype.storm.spout.ISpoutWaitStrategy



> On Aug 15, 2017, at 11:34, Mahak Goel <mahakgoel94@gmail.com> wrote:
> 
> I tried adding TOPOLOGY_SPOUT_WAIT_STRATEGY  and TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS
in the spouts config but that didn't seem to have an effect. 
> 
> 
>> On Aug 15, 2017, at 11:28, Mahak Goel <mahakgoel94@gmail.com> wrote:
>> 
>> Hi Stig, 
>> 
>> Thank you. However it looks like from this post there is a way to do it on a per
spout basis. 
>> https://groups.google.com/forum/m/#!search/Storm$200.8.1$20released/storm-user/hVbXtBdCkQo
>> 
>> Do you or does anyone else know if this is still a possibility? If so, how do I do
it? 
>> 
>> 
>>> On Aug 15, 2017, at 11:14, Stig Rohde Døssing <srdo@apache.org> wrote:
>>> 
>>> Hi Mahak,
>>> 
>>> I haven't checked in any detail, but I suspect there isn't. I'd have said you
could set the configuration for the spout via the SpoutDeclarer addConfiguration methods when
declaring the spout, but it looks like the wait strategy and backoff are both removed from
the component configuration, and only read from the topology level configuration https://github.com/apache/storm/blob/64e29f365c9b5d3e15b33f33ab64e200345333e4/storm-client/src/jvm/org/apache/storm/executor/Executor.java#L431.

>>> 
>>> 2017-08-15 16:45 GMT+02:00 Brian Taylor <brian@resolvingarchitecture.com>:
>>>> Unsubscribe
>>>> 
>>>> Sent from BlueMail
>>>>> On Aug 15, 2017, at 10:34 AM, Mahak Goel <mahakgoel94@gmail.com>
wrote:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> I know I can configure a sleep wait strategy in the defaults.yaml and
that will apply to all spouts in the topology. Is there a way to do this on a spout by spout
basis? That is, is there a way to configure different times for different spouts? 
>>>>> 
>>>>> Thanks!
>>> 

Mime
View raw message