spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chen Song <chen.song...@gmail.com>
Subject Access batch statistics in Spark Streaming
Date Mon, 08 Feb 2016 16:34:49 GMT
Apologize in advance if someone has already asked and addressed this
question.

In Spark Streaming, how can I programmatically get the batch statistics
like schedule delay, total delay and processing time (They are shown in the
job UI streaming tab)? I need such information to raise alerts in some
circumstances. For example, if the scheduling is delayed more than a
threshold.

Thanks,
Chen

Mime
View raw message