spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tathagata Das (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-1386) Spark Streaming UI
Date Tue, 01 Apr 2014 20:48:14 GMT
Tathagata Das created SPARK-1386:
------------------------------------

             Summary: Spark Streaming UI
                 Key: SPARK-1386
                 URL: https://issues.apache.org/jira/browse/SPARK-1386
             Project: Spark
          Issue Type: New Feature
          Components: Streaming
            Reporter: Tathagata Das


When debugging Spark Streaming applications it is necessary to monitor certain metrics that
are not shown in the Spark application UI. For example, what is average processing time of
batches? What is the scheduling delay? Is the system able to process as fast as it is receiving
data? How many records I am receiving through my receivers? 

While the StreamingListener interface introduced in the 0.9 provided some of this information,
it could only be accessed programmatically. A UI that shows information specific to the streaming
applications is necessary for easier debugging.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message