spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-3801) Make app dir cleanup more efficient
Date Sun, 05 Oct 2014 14:19:33 GMT

    [ https://issues.apache.org/jira/browse/SPARK-3801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159536#comment-14159536
] 

Sean Owen commented on SPARK-3801:
----------------------------------

I think this change will have to be subsumed in a slightly larger change to fix https://issues.apache.org/jira/browse/SPARK-3794
since {{FileUtils}} shouldn't be used in Spark anyway.

> Make app dir cleanup more efficient
> -----------------------------------
>
>                 Key: SPARK-3801
>                 URL: https://issues.apache.org/jira/browse/SPARK-3801
>             Project: Spark
>          Issue Type: Improvement
>            Reporter: Liang-Chi Hsieh
>            Priority: Minor
>
> The newly merged more conservative app directory cleanup can be more efficient. Since
it is not needed to store newer files, using 'exists' instead of 'filter' can be more efficient
because redundant items are skipped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message