storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sam mohel <sammoh...@gmail.com>
Subject Re: simple question about grouping
Date Tue, 24 Jan 2017 06:47:58 GMT
thanks for the link it's so helpful

On Tue, Jan 24, 2017 at 7:31 AM, Navin Ipe <navin.ipe@searchlighthealth.com>
wrote:

> More specifically, the bottom of this page explains what the numbers mean
> in terms of tasks and executors: http://nrecursions.blogspot.
> in/2016/05/more-concepts-of-apache-storm-you-need.html
>
> On Mon, Jan 23, 2017 at 5:28 PM, sam mohel <sammohel5@gmail.com> wrote:
>
>> really thanks for your help
>>
>> On Mon, Jan 23, 2017 at 1:59 PM, Arun Mahadevan <arunm@apache.org> wrote:
>>
>>> There is no magic number, it depends on the specific problem you are
>>> trying to solve. You start with some reasonable value for the parallelism
>>> and tune it based on your requirements. You could also start with a higher
>>> number of “tasks” than the parallelism and then you can rebalance your
>>> topology and adjust parallelism on the fly to scale up or down.
>>>
>>>
>>>
>>> See the slides from Taylor’s “Scaling Storm” presentation, you might
>>> find it useful - http://www.slideshare.net/ptgo
>>> etz/scaling-apache-storm-strata-hadoopworld-2014
>>>
>>>
>>>
>>>
>>>
>>> *From: *sam mohel <sammohel5@gmail.com>
>>> *Reply-To: *"user@storm.apache.org" <user@storm.apache.org>
>>> *Date: *Monday, January 23, 2017 at 4:58 PM
>>>
>>> *To: *"user@storm.apache.org" <user@storm.apache.org>
>>> *Subject: *Re: simple question about grouping
>>>
>>>
>>>
>>> Many thanks , but how and when can i decide that this number is perfect
>>> form me or not ?
>>>
>>>
>>>
>>> On Mon, Jan 23, 2017 at 1:27 PM, Arun Mahadevan <arunm@apache.org>
>>> wrote:
>>>
>>> > builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout");
>>> i found this example but couldn't know why he use number 4 ?
>>>
>>>
>>>
>>> This is the “parallelism hint” (the number of threads) for “MyBolt”.
So
>>> in your example there will be 4 threads executing “MyBolt” across the
>>> workers in your cluster and the tuples from “MySpout” would be randomly
>>> distributed across all of the 4 instances of your bolt.
>>>
>>>
>>>
>>> Also see http://storm.apache.org/releases/1.0.1/Understanding-the-par
>>> allelism-of-a-Storm-topology.html
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *From: *sam mohel <sammohel5@gmail.com>
>>> *Reply-To: *"user@storm.apache.org" <user@storm.apache.org>
>>> *Date: *Monday, January 23, 2017 at 4:47 PM
>>> *To: *"user@storm.apache.org" <user@storm.apache.org>
>>> *Subject: *Re: simple question about grouping
>>>
>>>
>>>
>>> excuse me , if i have single spout and single bolt and the bolt doing 2
>>> process so can i do like this
>>> builder.setSpout("MySpout", new mySpout(), 1);
>>> builder.setBolt("MyBolt", new MyBolt(), 4).shuffleGrouping("MySpout"); i
>>> found this example but couldn't know why he use number 4 ?
>>>
>>>
>>>
>>> On Mon, Jan 23, 2017 at 1:13 PM, sam mohel <sammohel5@gmail.com> wrote:
>>>
>>> thanks for replying
>>>
>>>
>>>
>>> On Mon, Jan 23, 2017 at 1:14 PM, Arun Mahadevan <arunm@apache.org>
>>> wrote:
>>>
>>>
>>>
>>> Grouping makes sense only when you have more than one task for a bolt.
>>> If your bolt has more than one task, then the grouping will decide how the
>>> tuples from the spout are distributed to the individual tasks of the bolt.
>>> (shuffe = random, fields = keyed on some field and so on).
>>>
>>>
>>>
>>> See http://storm.apache.org/releases/current/Concepts.html
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Arun
>>>
>>>
>>>
>>>
>>>
>>> *From: *sam mohel <sammohel5@gmail.com>
>>> *Reply-To: *"user@storm.apache.org" <user@storm.apache.org>
>>> *Date: *Monday, January 23, 2017 at 3:09 PM
>>> *To: *"user@storm.apache.org" <user@storm.apache.org>, "
>>> dev@storm.apache.org" <dev@storm.apache.org>
>>> *Subject: *simple question about grouping
>>>
>>>
>>>
>>> i have text file contains data . size of this file is 3.5 MB . My
>>> topology consists of one spout and one bolt so is that possible to make all
>>> processing in one bolt and in this case what is the role of grouping here ?
>>>
>>> Thanks in advance
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>
>
> --
> Regards,
> Navin
>

Mime
View raw message