storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Weathers <eweath...@groupon.com>
Subject Re: Need a clear explanation
Date Wed, 27 Apr 2016 06:46:14 GMT
So, you got those quotes from the following page it seems (please cite your
source next time):

https://storm.apache.org/releases/current/FAQ.html

> *Number of workers a multiple of number of machines *

I'm not sure why that statement is there.  It entirely depends on how many
topologies you are running, what they are doing, etc.

> *Number of kafka partitions a multiple of number of spout parallelism*

This is a bit confusing, but the point is that a given kafka partition can
only be handled by a single spout executor.   You cannot have 2 spouts
reading from the same partition.   So if you have say 10 kafka partitions
for a given topic, then the number of spout executors (i.e., the
parallelism) should be 1, 2, 5, or 10. Numbers that are a factor of 10.
Otherwise you'll have unused executors.

- Erik

On Tue, Apr 26, 2016 at 9:07 PM, Sai Dilip Reddy Kiralam <
dkiralam@aadhya-analytics.com> wrote:

> can some one explain me this ?
>
>
>
> *Best regards,*
>
> *K.Sai Dilip Reddy.*
>
> On Mon, Apr 25, 2016 at 1:48 PM, Sai Dilip Reddy Kiralam <
> dkiralam@aadhya-analytics.com> wrote:
>
>> Hi,
>>
>> I'm confused in setting up workers to storm cluster.
>> please some one explain the following with example.
>>
>>
>> * "1) Number of workers a multiple of number of machines **  2) *
>>
>> *Number of kafka partitions a multiple of number of spout parallelism;"  *
>>
>>
>>
>>
>> *Best regards,*
>>
>> *K.Sai Dilip Reddy.*
>>
>
>

Mime
View raw message