storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Jackson <jasonj...@gmail.com>
Subject Re: JVM heap options for Supervisor
Date Wed, 09 Apr 2014 05:40:28 GMT
The supervisor calls downloadChunk on nimbus API
https://github.com/apache/incubator-storm/blob/master/storm-core/src/storm.thrifteach
chunk is persisted to local disk on the supervisor before the next
chunk is grabbed. So the whole jar does need to be stored in RAM.


On Tue, Apr 8, 2014 at 6:14 PM, Srinath C <srinath.c@gmail.com> wrote:

> Thanks for the reply Jason.
> Supervisor doesn't need it in the classpath. But to provide the classpath
> to the worker processes it must be transferring the jar from nimbus. Hence
> the question.
>
>
> On Wed, Apr 9, 2014 at 6:32 AM, Jason Jackson <jasonjckn@gmail.com> wrote:
>
>> supervisor JVM does not load topology jar in it's classpath. We use 256mb
>> heapsize, that ought to be fine.
>>
>>
>> On Sun, Apr 6, 2014 at 11:11 PM, Srinath C <srinath.c@gmail.com> wrote:
>>
>>> Hi,
>>>     I'm trying to figure out a reasonable amount of heap to grant to the
>>> supervisor process on the storm worker machines. What are the factors that
>>> must be considered? I'm thinking - size of the topology jar and number of
>>> slots should be considered. But would like to hear if anyone was able to
>>> figure out the least amount of heap that can be safely granted.
>>>
>>>     Also, does the supervisor fully load the topology jar into memory or
>>> are there any optimizations done?
>>>
>>> Thanks,
>>> Srinath.
>>>
>>>
>>
>

Mime
View raw message