hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10357) Memory Leak in UserGroupInformation.doAs for JDBC Connection to Hive
Date Wed, 05 Mar 2014 14:35:48 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13920892#comment-13920892

Daryn Sharp commented on HADOOP-10357:

This intrigued me enough to download the 1.x source and take a quick look.  There is indeed
a subtle bug but in practice it shouldn't be tickled.  Multiple initializations of the metrics
(which creates the UgiInstrumentation) should be prevented by the isInitialized boolean. 
This can only be bypassed by directly calling {{UGI.setConfiguration}} which the sample code
is not doing.  Maybe the metrics system is cloning the object??

Aside, the screenshot shows 1004 instances which is not equal to 200k?  Unless you aborted
early, this seems to indicate GC should eventually cleanup the instances.

> Memory Leak in UserGroupInformation.doAs for JDBC Connection to Hive
> --------------------------------------------------------------------
>                 Key: HADOOP-10357
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10357
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 1.2.0
>            Reporter: Larry McCay
>         Attachments: visualvm-hive.png
> When using UGI.doAs in order to make a connection there appears to be a memory leak involving
the UGI that is used for the doAs and the UGI held by TUGIAssumingTransport.
> When using this approach to establishing a JDBC connection in an environment that will
serve many users and requests client side eventually runs out of memory.

This message was sent by Atlassian JIRA

View raw message