spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (Jira)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-27819) Retry cleanup of disk persisted RDD via external shuffle service when it failed via executor
Date Mon, 16 Mar 2020 22:52:05 GMT

     [ https://issues.apache.org/jira/browse/SPARK-27819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dongjoon Hyun updated SPARK-27819:
----------------------------------
    Affects Version/s:     (was: 3.0.0)
                       3.1.0

> Retry cleanup of disk persisted RDD via external shuffle service when it failed via executor
> --------------------------------------------------------------------------------------------
>
>                 Key: SPARK-27819
>                 URL: https://issues.apache.org/jira/browse/SPARK-27819
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 3.1.0
>            Reporter: Attila Zsolt Piros
>            Priority: Major
>
> This issue is created for not losing the idea came up during SPARK-27677 (at org.apache.spark.storage.BlockManagerMasterEndpoint#removeRdd):
> {panel:title=Quote}
> In certain situations (e.g. executor death) it would make sense to retry this through
the shuffle service. But I'm not sure how to safely detect those situations (or whether it
makes sense to always retry through the shuffle service).
> {panel}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message