storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Spico Florin <spicoflo...@gmail.com>
Subject Re: GC overhead limit exceeded
Date Mon, 11 Apr 2016 09:02:23 GMT
Hi!
  For me is strange that even after GC the memory that you have on the heap
dashboard remains almost at the same level. Can you perform GC from the
button Perform GC and see if the memory from the heap decreases?
Also, go to profiler tab and see what you get from there. Have you put a
large dictionary (hashmap) on your spout/bolts? You can debug your topology
by running your code in LocalCluster  and see the size of your map.
Also, with what XmX and XMs are you running your JVM?

Regards,
  Florin

On Mon, Apr 11, 2016 at 7:31 AM, sam mohel <sammohel5@gmail.com> wrote:

>
> ​@spico there is what i got after ruuning code again in local mode , how
> can i know if there is memory leak or not ?
>
>
>
> On Fri, Apr 8, 2016 at 1:45 AM, sam mohel <sammohel5@gmail.com> wrote:
>
>> sorry it's supposed to use hashmap.remove() to not to make it reach to
>> heapsize  right !
>>
>> On Fri, Apr 8, 2016 at 1:43 AM, sam mohel <sammohel5@gmail.com> wrote:
>>
>>> Really thanks for your patience , what i got about hashmap you mentioned
>>> that it's supposed not to use hashmap.remove();
>>> Right ?
>>>
>>>
>>>
>>> On Thu, Apr 7, 2016 at 10:45 AM, Spico Florin <spicoflorin@gmail.com>
>>> wrote:
>>>
>>>> Hi!
>>>>   By release the hashmap, I mean that you need to remove the keys at
>>>> some points. Meaning hashMap.remove(key). If you just use hashMap.put() in
>>>> the nextTuple method of the spout or in the execute method of the bolt, and
>>>> never use the hashMap.remove() and your hashMap is a field in the Bolt or
>>>> Spout class, then your map will grow and you'll reach your heap Size.
>>>>  The issue that yo have with the jvisualvm is that you have installed
>>>> only the Java Runtime Evironment (only the java vm) but not the the JDK
>>>> (Java Development Kit) . Please install the JDK.
>>>> After installing look at hashmap classes. Check the memory size for
>>>> them. Run GC and check if the memort size for them grows. If they grow even
>>>> after GC then you could have a memory leak.
>>>>
>>>> I hope that it helps.
>>>>  Florin
>>>>
>>>>
>>>>
>>>> On Wed, Apr 6, 2016 at 8:49 AM, sam mohel <sammohel5@gmail.com> wrote:
>>>>
>>>>> @florin
>>>>> i used this command  java -XX:+PrintFlagsFinal -version | grep HeapSize
>>>>>
>>>>> and got
>>>>>
>>>>> uintx ErgoHeapSizeLimit                         = 0
>>>>> {product}
>>>>>     uintx HeapSizePerGCThread                       = 87241520
>>>>> {product}
>>>>>     uintx InitialHeapSize                          := 63056640
>>>>> {product}
>>>>>     uintx LargePageHeapSizeThreshold                = 134217728
>>>>> {product}
>>>>>     uintx MaxHeapSize                              := 1010827264
>>>>> {product}
>>>>>
>>>>> On Wed, Apr 6, 2016 at 12:44 AM, sam mohel <sammohel5@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> sorry i forgot to mention that my ram is 3.8 GB and i used hahsmap
in
>>>>>> the code but i don't know what do you mean by release it ?
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 6, 2016 at 12:20 AM, sam mohel <sammohel5@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> @ florin thanks for replying , i installed tool but got when
i ran
>>>>>>> it
>>>>>>> i checked update-alternatives --config java
>>>>>>> There are 3 choices for the alternative java (providing
>>>>>>> /usr/bin/java).
>>>>>>>
>>>>>>>   Selection    Path
>>>>>>> Priority   Status
>>>>>>> ------------------------------------------------------------
>>>>>>>   0            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
>>>>>>> 1071      auto mode
>>>>>>> * 1            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java
>>>>>>> 1061      manual mode
>>>>>>>   2            /usr/lib/jvm/java-6-oracle/jre/bin/java
>>>>>>> 1062      manual mode
>>>>>>>   3            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
>>>>>>> 1071      manual mode
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ​
>>>>>>>
>>>>>>> On Sun, Apr 3, 2016 at 9:19 PM, Spico Florin <spicoflorin@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> hi!
>>>>>>>> before increasing the ram (by rpoviding in command line arguments
>>>>>>>> the jvm options java -Xmx) try to use a profile tool such
as jvisualvm
>>>>>>>> jprobe to see if you have amemory leak. do you use a cache
(for example
>>>>>>>> hashmap where you store some data but never relese it). how
much ram do you
>>>>>>>> have on your machine? check your default heap size with the
help of this
>>>>>>>> link
>>>>>>>>
>>>>>>>> http://stackoverflow.com/questions/4667483/how-is-the-default-java-heap-size-determined
>>>>>>>> regards florin
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sunday, April 3, 2016, sam mohel <sammohel5@gmail.com>
wrote:
>>>>>>>> > do you mean in storm.yaml ? or where ?
>>>>>>>> >
>>>>>>>> > On Sun, Apr 3, 2016 at 11:56 AM, Andrey Dudin <
>>>>>>>> doodin201@gmail.com> wrote:
>>>>>>>> >>
>>>>>>>> >> Try to extra more RAM for this topology.
>>>>>>>> >> -Xms and -Xmx options
>>>>>>>> >>
>>>>>>>> >> 2016-04-03 1:32 GMT+03:00 sam mohel <sammohel5@gmail.com>:
>>>>>>>> >>>
>>>>>>>> >>> i'm facing problem with topology i ran it in
local mode and got
>>>>>>>> >>> Async loop died!java.lang.OutOfMemoryError:
GC overhead limit
>>>>>>>> exceeded
>>>>>>>> >>>
>>>>>>>> >>> Can you help on this ? if there is any data
you need for
>>>>>>>> helping just tell me
>>>>>>>> >>>
>>>>>>>> >>> Thanks in advance
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >> --
>>>>>>>> >> С уважением Дудин Андрей
>>>>>>>> >
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message