storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <mj...@apache.org>
Subject Re: topology.ackers set to 0, still acking in downstream bolts?
Date Mon, 04 Jan 2016 16:59:26 GMT
Hi John,

fair enough -- I was not aware that you can disable fault-tolerance this
way. Good to know :)

My answer is still the same. Calls to Collector.ack(...) are always
counted and show up in the UI, but *no* ack messages will be sent for
your case.

-Matthias

On 01/04/2016 02:10 AM, John Yost wrote:
> Hi Matthias,
> 
> Gotcha, OK, the reason I thought that setting the number of ackers to
> zero would turn off acking for any bolts downstream from the KafkaSpout
> is the following guidance
> on http://storm.apache.org/documentation/Guaranteeing-message-processing.html:
> 
> There are three ways to remove reliability. *The first is to set
> Config.TOPOLOGY_ACKERS to 0. In this case, Storm will call
> the |ack| method on the spout immediately after the spout emits a tuple.
> The tuple tree won't be tracked.*
> 
> The second way is to remove reliability on a message by message basis.
> You can turn off tracking for an individual spout tuple by omitting a
> message id in the |SpoutOutputCollector.emit| method.
> 
> Finally, if you don't care if a particular subset of the tuples
> downstream in the topology fail to be processed, you can emit them as
> unanchored tuples. Since they're not anchored to any spout tuples, they
> won't cause any spout tuples to fail if they aren't acked.
> 
> --John
> 
> 
> 
> 
> On Wed, Dec 23, 2015 at 5:01 PM, Matthias J. Sax <mjsax@apache.org
> <mailto:mjsax@apache.org>> wrote:
> 
>     Hi,
> 
>     if you want to disable fault-tolerance, you need to emit tuples in your
>     spout without message IDs
> 
>     -> collector.emit(new Values(...)); // no messageId provided
> 
>     The parameter "topology.acker.executors" sets the number of ackers you
>     want to use in your topology. I am not sure, if setting it to zero
>     really disables fault-tolerance. (I would assume that fault-tolerance is
>     still enabled and you get many failing tuples---due to timeout---because
>     there are no ackers that can process the incoming ack messages.
> 
>     In any case, if your Bolts call "collector.ack(...)" this will be
>     monitored in the UI. However, it does not necessarily mean that
>     fault-tolerance is enabled or disabled. If you don't want to see an ack
>     count in the UI, you need to remove the line of code in your
>     Bolt.execute() methods that do the actual "collector.ack(...)" call.
> 
>     If you see a value for "acked" and "Complete latency" in your spouts,
>     fault-tolerance is enabled. If both values stay zero (as well as
>     "Failed") fault-tolerance if disabled.
> 
>     -Matthias
> 
> 
>     On 12/23/2015 03:31 PM, John Yost wrote:
>     > Hi Everyone,
>     >
>     > I have a topology that consists of a KafkaSpout and three
>     downstream bolts.
>     >
>     > I turned off acking by setting topology.acker.executors to 0. It's my
>     > understanding that I should only have acking in the KafkaSpout.
>     However,
>     > I see acking in all of my downstream bolts as well. Any ideas as
>     to why?
>     >
>     > Thanks
>     >
>     > --John
> 
> 


Mime
View raw message