storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srinath C <srinat...@gmail.com>
Subject Re: JVM heap options for Supervisor
Date Wed, 09 Apr 2014 07:03:16 GMT
Thanks Jason!


On Wed, Apr 9, 2014 at 11:10 AM, Jason Jackson <jasonjckn@gmail.com> wrote:

> The supervisor calls downloadChunk on nimbus API
> https://github.com/apache/incubator-storm/blob/master/storm-core/src/storm.thrifteach
chunk is persisted to local disk on the supervisor before the next
> chunk is grabbed. So the whole jar does need to be stored in RAM.
>
>
> On Tue, Apr 8, 2014 at 6:14 PM, Srinath C <srinath.c@gmail.com> wrote:
>
>> Thanks for the reply Jason.
>> Supervisor doesn't need it in the classpath. But to provide the classpath
>> to the worker processes it must be transferring the jar from nimbus. Hence
>> the question.
>>
>>
>> On Wed, Apr 9, 2014 at 6:32 AM, Jason Jackson <jasonjckn@gmail.com>wrote:
>>
>>> supervisor JVM does not load topology jar in it's classpath. We use
>>> 256mb heapsize, that ought to be fine.
>>>
>>>
>>> On Sun, Apr 6, 2014 at 11:11 PM, Srinath C <srinath.c@gmail.com> wrote:
>>>
>>>> Hi,
>>>>     I'm trying to figure out a reasonable amount of heap to grant to
>>>> the supervisor process on the storm worker machines. What are the factors
>>>> that must be considered? I'm thinking - size of the topology jar and number
>>>> of slots should be considered. But would like to hear if anyone was able
to
>>>> figure out the least amount of heap that can be safely granted.
>>>>
>>>>     Also, does the supervisor fully load the topology jar into memory
>>>> or are there any optimizations done?
>>>>
>>>> Thanks,
>>>> Srinath.
>>>>
>>>>
>>>
>>
>

Mime
View raw message