storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Yost <hokiege...@gmail.com>
Subject Re: topology.ackers set to 0, still acking in downstream bolts?
Date Mon, 04 Jan 2016 01:10:35 GMT
Hi Matthias,

Gotcha, OK, the reason I thought that setting the number of ackers to zero
would turn off acking for any bolts downstream from the KafkaSpout is the
following guidance on
http://storm.apache.org/documentation/Guaranteeing-message-processing.html:

There are three ways to remove reliability. *The first is to set
Config.TOPOLOGY_ACKERS to 0. In this case, Storm will call the ack method
on the spout immediately after the spout emits a tuple. The tuple tree
won't be tracked.*

The second way is to remove reliability on a message by message basis. You
can turn off tracking for an individual spout tuple by omitting a message
id in the SpoutOutputCollector.emit method.

Finally, if you don't care if a particular subset of the tuples downstream
in the topology fail to be processed, you can emit them as unanchored
tuples. Since they're not anchored to any spout tuples, they won't cause
any spout tuples to fail if they aren't acked.

--John



On Wed, Dec 23, 2015 at 5:01 PM, Matthias J. Sax <mjsax@apache.org> wrote:

> Hi,
>
> if you want to disable fault-tolerance, you need to emit tuples in your
> spout without message IDs
>
> -> collector.emit(new Values(...)); // no messageId provided
>
> The parameter "topology.acker.executors" sets the number of ackers you
> want to use in your topology. I am not sure, if setting it to zero
> really disables fault-tolerance. (I would assume that fault-tolerance is
> still enabled and you get many failing tuples---due to timeout---because
> there are no ackers that can process the incoming ack messages.
>
> In any case, if your Bolts call "collector.ack(...)" this will be
> monitored in the UI. However, it does not necessarily mean that
> fault-tolerance is enabled or disabled. If you don't want to see an ack
> count in the UI, you need to remove the line of code in your
> Bolt.execute() methods that do the actual "collector.ack(...)" call.
>
> If you see a value for "acked" and "Complete latency" in your spouts,
> fault-tolerance is enabled. If both values stay zero (as well as
> "Failed") fault-tolerance if disabled.
>
> -Matthias
>
>
> On 12/23/2015 03:31 PM, John Yost wrote:
> > Hi Everyone,
> >
> > I have a topology that consists of a KafkaSpout and three downstream
> bolts.
> >
> > I turned off acking by setting topology.acker.executors to 0. It's my
> > understanding that I should only have acking in the KafkaSpout. However,
> > I see acking in all of my downstream bolts as well. Any ideas as to why?
> >
> > Thanks
> >
> > --John
>
>

Mime
View raw message