spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (SPARK-1202) Add a "cancel" button in the UI for stages
Date Sat, 29 Mar 2014 22:40:14 GMT


ASF GitHub Bot commented on SPARK-1202:

Github user kayousterhout commented on the pull request:
    It looks like github is just moving slowly today...the commit just got pulled in.  I took
another look at this and have a question: what happens for stages that are used for multiple
jobs?  Right now, stageIdToJobId in the UI code you added just maps a stage to a single job
id.  So, if stage0 is used by JobA and jobB, the ui code only stores one of these jobs, and
then cancelJob() will only be called for one of the jobs.  cancelJob() ultimately calls DAGScheduler.handleJobCancellation(),
which only cancels the stages that are independent to the job.  So, because stage0 is not
independent to either of the jobs, it won't get cancelled.  Did I misunderstand this?

> Add a "cancel" button in the UI for stages
> ------------------------------------------
>                 Key: SPARK-1202
>                 URL:
>             Project: Apache Spark
>          Issue Type: New Feature
>          Components: Web UI
>            Reporter: Patrick Cogan
>            Assignee: Sundeep Narravula
>            Priority: Critical
>             Fix For: 1.0.0
> Seems like this would be really useful for people. It's not that hard, we just need to
lookup the jobs associated with the stage and kill them. Might involve exposing some additional
API's in SparkContext.

This message was sent by Atlassian JIRA

View raw message