storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nathan Leung <ncle...@gmail.com>
Subject Re: Manage parallelism in Storm
Date Mon, 23 Feb 2015 19:21:33 GMT
The docs have a good discussion of how acks work:
https://storm.apache.org/documentation/Guaranteeing-message-processing.html

On Mon, Feb 23, 2015 at 2:14 PM, Nick R. Katsipoulakis <
nick.katsip@gmail.com> wrote:

> Ok, that makes sense. However, I do not understand how ackers work. Are
> they spouts/bolts exactly the same as my original spouts/bolts and they
> perform different functions? Also, is there any way in my code to
> differentiate the behavior of a spout/bolt with its acker?
>
> Thanks,
> Nick
>
> 2015-02-23 14:09 GMT-05:00 Nathan Leung <ncleung@gmail.com>:
>
> with 4 workers, by default, you would also have 4 acker tasks (1 /
>> worker), which is why you might see 8 tasks.
>>
>> On Mon, Feb 23, 2015 at 1:56 PM, Nick R. Katsipoulakis <
>> nick.katsip@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I am trying to control my cluster's parallelism and I want to achieve
>>> the following: 1 worker per component, and 1 executioner per worker and 1
>>> task per executioner. In order to achieve the former, I submit my
>>> components to the topology as follows:
>>>
>>> Config conf = new Config();
>>>
>>> TopologyBuilder builder = new TopologyBuilder();
>>> builder.setSpout("spout_1",
>>>                 new GreenSpout(), 1)
>>>                 .setNumTasks(1);
>>> builder.setBolt("bolt_1",
>>>                 new GreenBolt(), 1)
>>>                 .setNumTasks(1)
>>>                 .directGrouping("spout_1");
>>> builder.setBolt("bolt_2",
>>>                 new RedBolt(), 1)
>>>                 .setNumTasks(1)
>>>                 .directGrouping("bolt_1");
>>> builder.setBolt("bolt_3",
>>>                 new YellowBolt(), 1)
>>>                 .setNumTasks(1)
>>>                 .directGrouping("bolt_2");
>>>
>>> conf.setDebug(true);
>>> conf.setNumWorkers(4);
>>> StormSubmitter.submitTopology("experimental-top", conf,
>>> builder.createTopology());
>>>
>>> The above not only sets the parallelism hint (executioner threads per
>>> worker) to 1, but also the number of workers per component to 1. However,
>>> when I submit my topology, and I perform a storm list, I can see that I
>>> have 4 workers and 8 tasks running. Is this normal? My goal is to have each
>>> component executed by 1 thread on each machine. Am I missing something?
>>>
>>> Thanks,
>>> Nick
>>>
>>>
>>> --
>>> Nikolaos Romanos Katsipoulakis,
>>> University of Pittsburgh, PhD candidate
>>>
>>
>>
>
>
> --
> Nikolaos Romanos Katsipoulakis,
> University of Pittsburgh, PhD candidate
>

Mime
View raw message