spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Wendell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-1432) Potential memory leak in stageIdToExecutorSummaries in JobProgressTracker
Date Mon, 07 Apr 2014 17:08:15 GMT

    [ https://issues.apache.org/jira/browse/SPARK-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962022#comment-13962022
] 

Patrick Wendell commented on SPARK-1432:
----------------------------------------

https://github.com/apache/spark/pull/338

> Potential memory leak in stageIdToExecutorSummaries in JobProgressTracker
> -------------------------------------------------------------------------
>
>                 Key: SPARK-1432
>                 URL: https://issues.apache.org/jira/browse/SPARK-1432
>             Project: Spark
>          Issue Type: Bug
>          Components: Web UI
>    Affects Versions: 0.9.0
>            Reporter: Davis Shepherd
>            Assignee: Davis Shepherd
>             Fix For: 1.0.0, 0.9.2
>
>
> JobProgressTracker continuously cleans up old metadata as per the spark.ui.retainedStages
configuration parameter. It seems however that not all metadata maps are being cleaned, in
particular stageIdToExecutorSummaries could grow in an unbounded manner in a long running
application.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message