hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naveen Gangam (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13749) Memory leak in Hive Metastore
Date Fri, 01 Jul 2016 20:47:11 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359603#comment-15359603
] 

Naveen Gangam commented on HIVE-13749:
--------------------------------------

Have a new heap dump from a customer that has been running for an extended period of time,
with the following properties removed.
{code}
<property><name>hive.compactor.initiator.on</name><value>true</value></property>
<property><name>hive.compactor.worker.threads</name><value>1</value></property>
{code}

The leak from the above compactor thread was later addressed in HIVE-13151 that the customer
was not running with. So I had them remove these properties.

HMS has been stable, no OOM crashes yet. There are about 214 instances of Configuration occupying
66MB of heap space, roughly 18% of the total active heap. Each of this instance was from an
individual thread holding on to the object in the threadlocals. I think we can save this unnecessary
reference and do better by cleaning up the threadlocal before the thread is returning to the
pool.

I have uploaded a new patch that does just that. Let me know of any thoughts. Thanks

> Memory leak in Hive Metastore
> -----------------------------
>
>                 Key: HIVE-13749
>                 URL: https://issues.apache.org/jira/browse/HIVE-13749
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 1.1.0
>            Reporter: Naveen Gangam
>            Assignee: Naveen Gangam
>         Attachments: HIVE-13749.1.patch, HIVE-13749.patch, Top_Consumers7.html
>
>
> Looking a heap dump of 10GB, a large number of Configuration objects(> 66k instances)
are being retained. These objects along with its retained set is occupying about 95% of the
heap space. This leads to HMS crashes every few days.
> I will attach an exported snapshot from the eclipse MAT.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message