storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Navin Ipe <navin....@searchlighthealth.com>
Subject Re: simple question about grouping
Date Tue, 24 Jan 2017 05:31:08 GMT
More specifically, the bottom of this page explains what the numbers mean
in terms of tasks and executors:
http://nrecursions.blogspot.in/2016/05/more-concepts-of-apache-storm-you-need.html

On Mon, Jan 23, 2017 at 5:28 PM, sam mohel <sammohel5@gmail.com> wrote:

> really thanks for your help
>
> On Mon, Jan 23, 2017 at 1:59 PM, Arun Mahadevan <arunm@apache.org> wrote:
>
>> There is no magic number, it depends on the specific problem you are
>> trying to solve. You start with some reasonable value for the parallelism
>> and tune it based on your requirements. You could also start with a higher
>> number of “tasks” than the parallelism and then you can rebalance your
>> topology and adjust parallelism on the fly to scale up or down.
>>
>>
>>
>> See the slides from Taylor’s “Scaling Storm” presentation, you might find
>> it useful - http://www.slideshare.net/ptgoetz/scaling-apache-storm-strat
>> a-hadoopworld-2014
>>
>>
>>
>>
>>
>> *From: *sam mohel <sammohel5@gmail.com>
>> *Reply-To: *"user@storm.apache.org" <user@storm.apache.org>
>> *Date: *Monday, January 23, 2017 at 4:58 PM
>>
>> *To: *"user@storm.apache.org" <user@storm.apache.org>
>> *Subject: *Re: simple question about grouping
>>
>>
>>
>> Many thanks , but how and when can i decide that this number is perfect
>> form me or not ?
>>
>>
>>
>> On Mon, Jan 23, 2017 at 1:27 PM, Arun Mahadevan <arunm@apache.org> wrote:
>>
>> > builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout");
>> i found this example but couldn't know why he use number 4 ?
>>
>>
>>
>> This is the “parallelism hint” (the number of threads) for “MyBolt”. So
>> in your example there will be 4 threads executing “MyBolt” across the
>> workers in your cluster and the tuples from “MySpout” would be randomly
>> distributed across all of the 4 instances of your bolt.
>>
>>
>>
>> Also see http://storm.apache.org/releases/1.0.1/Understanding-the-
>> parallelism-of-a-Storm-topology.html
>>
>>
>>
>>
>>
>>
>>
>> *From: *sam mohel <sammohel5@gmail.com>
>> *Reply-To: *"user@storm.apache.org" <user@storm.apache.org>
>> *Date: *Monday, January 23, 2017 at 4:47 PM
>> *To: *"user@storm.apache.org" <user@storm.apache.org>
>> *Subject: *Re: simple question about grouping
>>
>>
>>
>> excuse me , if i have single spout and single bolt and the bolt doing 2
>> process so can i do like this
>> builder.setSpout("MySpout", new mySpout(), 1);
>> builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i
>> found this example but couldn't know why he use number 4 ?
>>
>>
>>
>> On Mon, Jan 23, 2017 at 1:13 PM, sam mohel <sammohel5@gmail.com> wrote:
>>
>> thanks for replying
>>
>>
>>
>> On Mon, Jan 23, 2017 at 1:14 PM, Arun Mahadevan <arunm@apache.org> wrote:
>>
>>
>>
>> Grouping makes sense only when you have more than one task for a bolt. If
>> your bolt has more than one task, then the grouping will decide how the
>> tuples from the spout are distributed to the individual tasks of the bolt.
>> (shuffe = random, fields = keyed on some field and so on).
>>
>>
>>
>> See http://storm.apache.org/releases/current/Concepts.html
>>
>>
>>
>> Thanks,
>>
>> Arun
>>
>>
>>
>>
>>
>> *From: *sam mohel <sammohel5@gmail.com>
>> *Reply-To: *"user@storm.apache.org" <user@storm.apache.org>
>> *Date: *Monday, January 23, 2017 at 3:09 PM
>> *To: *"user@storm.apache.org" <user@storm.apache.org>, "
>> dev@storm.apache.org" <dev@storm.apache.org>
>> *Subject: *simple question about grouping
>>
>>
>>
>> i have text file contains data . size of this file is 3.5 MB . My
>> topology consists of one spout and one bolt so is that possible to make all
>> processing in one bolt and in this case what is the role of grouping here ?
>>
>> Thanks in advance
>>
>>
>>
>>
>>
>>
>>
>
>


-- 
Regards,
Navin

Mime
View raw message