spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-10370) After a stages map outputs are registered, all running attempts should be marked as zombies
Date Tue, 01 Sep 2015 09:20:46 GMT

    [ https://issues.apache.org/jira/browse/SPARK-10370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14725043#comment-14725043
] 

Apache Spark commented on SPARK-10370:
--------------------------------------

User 'suyanNone' has created a pull request for this issue:
https://github.com/apache/spark/pull/8550

> After a stages map outputs are registered, all running attempts should be marked as zombies
> -------------------------------------------------------------------------------------------
>
>                 Key: SPARK-10370
>                 URL: https://issues.apache.org/jira/browse/SPARK-10370
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.5.0
>            Reporter: Imran Rashid
>
> Follow up to SPARK-5259.  During stage retry, its possible for a stage to "complete"
by registering all its map output and starting the downstream stages, before the latest task
set has completed.  This will result in the earlier task set continuing to submit tasks, that
are both unnecessary and increase the chance of hitting SPARK-8029.
> Spark should mark all tasks sets for a stage as zombie as soon as its map output is registered.
 Note that this involves coordination between the various scheduler components ({{DAGScheduler}}
and {{TaskSetManager}} at least) which isn't easily testable with the current setup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message