spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aarondav <>
Subject [GitHub] incubator-spark pull request: add event listener when executors ar...
Date Mon, 17 Feb 2014 03:18:31 GMT
Github user aarondav commented on the pull request:
    What is the use-case you have in mind here? Just some sort of final status of all executors
right before terminating a job/shell?
    If you're just interested in the HDFS stats, you might take a look at our [MetricsSystem](,
where we do register [hdfs bytes_read](
(sorry, this part of the code is a little hairy to follow). If you can attach to our metrics
sink, you may be able to get the info you're looking for without modification to Spark.

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at or file a JIRA ticket with INFRA.

View raw message