storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Dudin <doodin...@gmail.com>
Subject Re: How to set number of slots in storm.yaml
Date Tue, 12 Apr 2016 08:54:27 GMT
Hi,

It's like virtualization. If you have many small topologies that need few
CPU resources, you can add more processes per CPU. It you have big
topologies that need a lot of CPU resources, you need to decrease number of
process per cpu. I advise to start from 1.2-1.5 process per CPU and compare
with 1 process per cpu and 2 process per cpu.


2016-04-12 7:44 GMT+03:00 Sai Dilip Reddy Kiralam <
dkiralam@aadhya-analytics.com>:

> Hi guys,
>
> Thank you for your suggestions but I'm not clear with the
> explanation.andrey please let me know one thing '*what do you mean by
> "depend by code"* ' ? Depend on code in the sense does it depends on
> number of bolts and spouts I'm using ?if yes can you give me a direct
> answer using an example having number of bolts and spouts.How many
> supervisor.slots should I consider in 16 core cpu?Answer to my question
> Still remains as a lock.
> machine.
>
>
>
> *Best regards,*
>
> *K.Sai Dilip Reddy.*
>
> On Mon, Apr 11, 2016 at 9:29 PM, Andrey Dudin <doodin201@gmail.com> wrote:
>
>> Hi Nikos.
>>
>> Yes, you are right, but sometimes java application show better
>> performance under high load when used 1.2-1.5 process per core. Thicker
>> loading is not always bad.
>>
>> 2016-04-11 18:23 GMT+03:00 Nikos R. Katsipoulakis <nick.katsip@gmail.com>
>> :
>>
>>> Hello both,
>>>
>>> Andrey, I believe its better to use one executor thread per core and
>>> only one worker process per socket. This way, the executor threads
>>> communicate with each other using LMAX Disruptor queues (therefore less
>>> latency) and Netty is used only when a message needs to be sent to another
>>> executor thread on a different machine.
>>>
>>> Also, if you need a lot of memory, you can start your worker process
>>> with as much Heap memory you want.
>>>
>>> Cheers,
>>> Nikos
>>>
>>> On Mon, Apr 11, 2016 at 9:15 AM, Andrey Dudin <doodin201@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>> It's depend by code. Usually starting from 1-1,5 process per core.
>>>>
>>>> 2016-04-11 16:10 GMT+03:00 Sai Dilip Reddy Kiralam <
>>>> dkiralam@aadhya-analytics.com>:
>>>>
>>>>>
>>>>> Hi All,
>>>>>
>>>>> I'm using a 16 core cpu as my supervisor node.My doubt is How many
>>>>> supervisor.slots should I set in supervisor storm.yaml conf file ?Is
there
>>>>> any formula to set no of slots based on the cpu core.
>>>>>
>>>>> Thanks in Advance
>>>>>
>>>>>
>>>>> *Best regards,*
>>>>>
>>>>> *K.Sai Dilip Reddy.*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> С уважением Дудин Андрей
>>>>
>>>
>>>
>>>
>>> --
>>> Nikos R. Katsipoulakis,
>>> Department of Computer Science
>>> University of Pittsburgh
>>>
>>
>>
>>
>> --
>> С уважением Дудин Андрей
>>
>
>


-- 
С уважением Дудин Андрей

Mime
View raw message