spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Vogelbacher (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-24437) Memory leak in UnsafeHashedRelation
Date Thu, 01 Nov 2018 14:44:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-24437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16671686#comment-16671686
] 

David Vogelbacher edited comment on SPARK-24437 at 11/1/18 2:43 PM:
--------------------------------------------------------------------

Hey [~mgaido], I am seeing something similar in one of our long running applications.
The longer it runs the higher heap usage in the driver grows. Taking a heap dump, most of
its space is taken up by {{UnsafeHashedRelation}} instances. Looking at their paths to GC,
they are strongly referenced by {{InMemoryRelation}} instances:
 !Screen Shot 2018-11-01 at 10.38.30 AM.png! 

So it looks like if dataframes are cached then the {{UnsafeHashedRelation}} instances are
never cleaned up, because they are strongly references by generated codegen plan? 
The context cleaner can only clean them up when they are no longer strongly referenced.


was (Author: dvogelbacher):
Hey [~mgaido], I am seeing something similar in one of our long running applications.
The longer it runs the higher heap usage in the driver grows. Taking a heap dump, most of
its space is taken up by {{UnsafeHashedRelation}} instances. Looking at their paths to GC,
they are strongly referenced by {{InMemoryRelation}} instances:
 !Screen Shot 2018-11-01 at 10.38.30 AM.png! 

So it looks like if dataframes are cached the {{UnsafeHashedRelation}}s are never cleaned
up, because they are strongly references by generated codegen plan? 
The context cleaner can only clean them up when they are no longer strongly referenced.

> Memory leak in UnsafeHashedRelation
> -----------------------------------
>
>                 Key: SPARK-24437
>                 URL: https://issues.apache.org/jira/browse/SPARK-24437
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.2.0
>            Reporter: gagan taneja
>            Priority: Critical
>         Attachments: Screen Shot 2018-05-30 at 2.05.40 PM.png, Screen Shot 2018-05-30
at 2.07.22 PM.png, Screen Shot 2018-11-01 at 10.38.30 AM.png
>
>
> There seems to memory leak with org.apache.spark.sql.execution.joins.UnsafeHashedRelation
> We have a long running instance of STS.
> With each query execution requiring Broadcast Join, UnsafeHashedRelation is getting
added for cleanup in ContextCleaner. This reference of UnsafeHashedRelation is being held
at some other Collection and not becoming eligible for GC and because of this ContextCleaner
is not able to clean it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message