storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From clay teahouse <clayteaho...@gmail.com>
Subject Re: questions on task, threads and workers
Date Mon, 02 Feb 2015 23:44:33 GMT
According to the storm docs, as of Storm 0.8 the*parallelism_hint* parameter
now specifies the initial number of executors (not tasks!) for that bolt.
I assume this means that number of tasks and the number of parallelism
hints have to be set separately and the number of executor threads (i.e.,
the parallelism hint) does not set the value of the number of tasks. Hence
my question, what happens if the number of tasks is less than the number of
executors?


On Mon, Feb 2, 2015 at 5:37 PM, Kosala Dissanayake <umaradissa@gmail.com>
wrote:

> 1. The default number of tasks for an executor thread is 1. So if you just
> specify the parallelism for a bolt as 3, you will have 3 executor threads.
> Since the default num of tasks for an executor thread is 1, this means you
> will get 3 tasks.
>
> You can increase the number of tasks for an executor thread from the
> default 1 by using setNumTasks.
>
> 2. I don't think that's possible.
>
> 3. No. The number of workers is just the number of processes to which
> executors can be allocated. If you have less workers, more executors will
> be assigned to each worker and vice versa.
>
>
>
>
> http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/
is
> a pretty good introduction to these concepts
>
> On Mon, Feb 2, 2015 at 9:54 AM, clay teahouse <clayteahouse@gmail.com>
> wrote:
>
>> Hi,
>> I have a few simple questions.
>> 1)In storm .9.x, what is the default value for the bolt num tasks?
>> According to the docs, the parallelism hint no longer sets the number of
>> tasks, but the number of executor threads.
>> 2)What happens if the number of tasks is less than the number of threads?
>> Should I assume this results in idle threads?
>> 3)Does the number of workers multiplies the number of tasks and threads?
>>
>> feedback appreciated,
>> Clay
>>
>
>

Mime
View raw message