spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-25285) Add executor task metrics to track the number of tasks started and of tasks successfully completed
Date Thu, 30 Aug 2018 19:48:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-25285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-25285:
------------------------------------

    Assignee:     (was: Apache Spark)

> Add executor task metrics to track the number of tasks started and of tasks successfully
completed
> --------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-25285
>                 URL: https://issues.apache.org/jira/browse/SPARK-25285
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 2.4.0
>            Reporter: Luca Canali
>            Priority: Minor
>
> The motivation for these additional metrics is to help in troubleshooting situations
when tasks fail, are killed and/or restarted. Currently available metrics include executor
threadpool metrics for task completed and for active tasks. The addition of threadpool tasStarted
metric will allow for example to collect info on the (approximate) number of failed tasks
by computing the difference thread started – (active threads + completed tasks and/or successfully
completed tasks).
> The proposed metric successfulTasks is also intended for this type of troubleshooting.
The difference between  successfulTasks and threadpool.completeTasks, is that the latter
is a (dropwizard library) gauge taken from the threadpool, while the former is a (dropwizard)
counter computed in the [[Executor]] class, when a task successfully completes, together with
several other task metrics counters.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message