storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick R. Katsipoulakis" <nick.kat...@gmail.com>
Subject Manage parallelism in Storm
Date Mon, 23 Feb 2015 18:56:30 GMT
Hello,

I am trying to control my cluster's parallelism and I want to achieve the
following: 1 worker per component, and 1 executioner per worker and 1 task
per executioner. In order to achieve the former, I submit my components to
the topology as follows:

Config conf = new Config();

TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("spout_1",
                new GreenSpout(), 1)
                .setNumTasks(1);
builder.setBolt("bolt_1",
                new GreenBolt(), 1)
                .setNumTasks(1)
                .directGrouping("spout_1");
builder.setBolt("bolt_2",
                new RedBolt(), 1)
                .setNumTasks(1)
                .directGrouping("bolt_1");
builder.setBolt("bolt_3",
                new YellowBolt(), 1)
                .setNumTasks(1)
                .directGrouping("bolt_2");

conf.setDebug(true);
conf.setNumWorkers(4);
StormSubmitter.submitTopology("experimental-top", conf,
builder.createTopology());

The above not only sets the parallelism hint (executioner threads per
worker) to 1, but also the number of workers per component to 1. However,
when I submit my topology, and I perform a storm list, I can see that I
have 4 workers and 8 tasks running. Is this normal? My goal is to have each
component executed by 1 thread on each machine. Am I missing something?

Thanks,
Nick


-- 
Nikolaos Romanos Katsipoulakis,
University of Pittsburgh, PhD candidate

Mime
View raw message