storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stig Rohde Døssing <s...@apache.org>
Subject Re: Spout sleep wait strategy
Date Tue, 15 Aug 2017 15:51:58 GMT
I think I might have misread the code. It looks like the method I linked
does the opposite of what I thought, and removes only the configuration
that is not listed in the link. I would expect using
SpoutDeclarer.addConfiguration to work then.

2017-08-15 17:36 GMT+02:00 Mahak Goel <mahakgoel94@gmail.com>:

> Text from post.
>
> 2. Spout wait strategies: There's two situations in which a spout needs to
> wait. The first is when the max spout pending limit is reached. The second
> is when nothing is emitted from nextTuple. Previously, Storm would just
> have that spout sit in a busy loop in those cases. What Storm does in those
> situations is now pluggable, and the default is now for the spout to sleep
> for 1 ms. This will cause the spout to use dramatically less CPU when it
> hits those cases, and it also obviates the need for spouts to do any
> sleeping in their implementation to be "polite". The wait strategy can be
> configured with TOPOLOGY_SPOUT_WAIT_STRATEGY and can be configured on a
> spout by spout basis. The interface to implement for a wait strategy is
> backtype.storm.spout.ISpoutWaitStrategy
>
>
>
> On Aug 15, 2017, at 11:34, Mahak Goel <mahakgoel94@gmail.com> wrote:
>
> I tried adding TOPOLOGY_SPOUT_WAIT_STRATEGY  and TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS
> in the spouts config but that didn't seem to have an effect.
>
>
> On Aug 15, 2017, at 11:28, Mahak Goel <mahakgoel94@gmail.com> wrote:
>
> Hi Stig,
>
> Thank you. However it looks like from this post there is a way to do it on
> a per spout basis.
> https://groups.google.com/forum/m/#!search/Storm$200.8.
> 1$20released/storm-user/hVbXtBdCkQo
>
> Do you or does anyone else know if this is still a possibility? If so, how
> do I do it?
>
>
> On Aug 15, 2017, at 11:14, Stig Rohde Døssing <srdo@apache.org> wrote:
>
> Hi Mahak,
>
> I haven't checked in any detail, but I suspect there isn't. I'd have said
> you could set the configuration for the spout via the SpoutDeclarer
> addConfiguration methods when declaring the spout, but it looks like the
> wait strategy and backoff are both removed from the component
> configuration, and only read from the topology level configuration
> https://github.com/apache/storm/blob/64e29f365c9b5d3e15b33f33ab64e2
> 00345333e4/storm-client/src/jvm/org/apache/storm/executor/
> Executor.java#L431.
>
> 2017-08-15 16:45 GMT+02:00 Brian Taylor <brian@resolvingarchitecture.com>:
>
>> Unsubscribe
>>
>> Sent from BlueMail <http://www.bluemail.me/r?b=9660>
>> On Aug 15, 2017, at 10:34 AM, Mahak Goel <mahakgoel94@gmail.com> wrote:
>>>
>>>
>>> Hi,
>>>
>>> I know I can configure a sleep wait strategy in the defaults.yaml and that will
apply to all spouts in the topology. Is there a way to do this on a spout by spout basis?
That is, is there a way to configure different times for different spouts?
>>>
>>> Thanks!
>>>
>>>
>

Mime
View raw message