storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nathan Leung <ncle...@gmail.com>
Subject Re: Manage parallelism in Storm
Date Mon, 23 Feb 2015 19:09:09 GMT
with 4 workers, by default, you would also have 4 acker tasks (1 / worker),
which is why you might see 8 tasks.

On Mon, Feb 23, 2015 at 1:56 PM, Nick R. Katsipoulakis <
nick.katsip@gmail.com> wrote:

> Hello,
>
> I am trying to control my cluster's parallelism and I want to achieve the
> following: 1 worker per component, and 1 executioner per worker and 1 task
> per executioner. In order to achieve the former, I submit my components to
> the topology as follows:
>
> Config conf = new Config();
>
> TopologyBuilder builder = new TopologyBuilder();
> builder.setSpout("spout_1",
>                 new GreenSpout(), 1)
>                 .setNumTasks(1);
> builder.setBolt("bolt_1",
>                 new GreenBolt(), 1)
>                 .setNumTasks(1)
>                 .directGrouping("spout_1");
> builder.setBolt("bolt_2",
>                 new RedBolt(), 1)
>                 .setNumTasks(1)
>                 .directGrouping("bolt_1");
> builder.setBolt("bolt_3",
>                 new YellowBolt(), 1)
>                 .setNumTasks(1)
>                 .directGrouping("bolt_2");
>
> conf.setDebug(true);
> conf.setNumWorkers(4);
> StormSubmitter.submitTopology("experimental-top", conf,
> builder.createTopology());
>
> The above not only sets the parallelism hint (executioner threads per
> worker) to 1, but also the number of workers per component to 1. However,
> when I submit my topology, and I perform a storm list, I can see that I
> have 4 workers and 8 tasks running. Is this normal? My goal is to have each
> component executed by 1 thread on each machine. Am I missing something?
>
> Thanks,
> Nick
>
>
> --
> Nikolaos Romanos Katsipoulakis,
> University of Pittsburgh, PhD candidate
>

Mime
View raw message