storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Schonfeld <downwindab...@gmail.com>
Subject Re: Possible bug in acker.clj or my lack of understanding in RotatingMap
Date Tue, 23 Dec 2014 15:20:28 GMT
Nathan,

Is that guarded in executor.clj by the (when spout-id clause (on 441 on
v0.9.3 tag)? I'm assuming it would all be null since (.remove pending id)
would return back bulk after the first.

Is my reading / understanding correct?

Thanks much! It's interesting to look at the internals and see how our
topologies are powered.

Daniel

On Tuesday, December 23, 2014, Nathan Marz <nathan@nathanmarz.com> wrote:

> The spout will ignore any acks or fails after the first one to account for
> this possibility.
>
> On Mon, Dec 22, 2014 at 8:10 AM, Daniel Schonfeld <downwindabeam@gmail.com
> <javascript:_e(%7B%7D,'cvml','downwindabeam@gmail.com');>> wrote:
>>
>> Hello,
>>
>> I am trying to understand some of the storm code base and I've run into
>> this question regarding the use of RotatingMap in acker.clj and
>> executor.clj.  It's probably my lack of understand of how the code works,
>> but i figured there is no harm in asking.
>>
>> So I see that in the spout, the rotating map combined with a tick tuple
>> set to TOPOLOGY-MESSAGE-TIMEOUT-SECS is used to timeout the tuple at the
>> spout subsequently calling the fail() method.
>>
>> I also see that inside the acker the same mechanism is used.
>>
>> The question is what happens when the spout fails the tuple from a
>> timeout, but the bolts continue processing the tuple's children so part of
>> the tuple DAG.  They keep sending ACK messages to the acker and update the
>> key in the RotatingMap there by causing the tuple to not timeout at the
>> acker even though it already did at the spout.
>>
>> Is there a possibility of receiving both a fail() and an ack() at the
>> *spout* given the above condition?
>>
>> Thanks!
>> Daniel
>>
>
>
> --
> Twitter: @nathanmarz
> http://nathanmarz.com
>

Mime
View raw message