spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ai Deng <wx.london....@gmail.com>
Subject reuse the Spark SQL internal metrics
Date Tue, 30 Aug 2016 21:17:54 GMT
Hi there,

I think the metrics inside of the different SparkPlan (like "numOutputRows"
in FilterExec) are useful to build any Dev dashboard or business monitoring.
Are there a easy way or exist solution to expose and persist these metrics
out of Spark UI (ex: send to Graphite)? Currently they are all "private"
inside Spark library.

And the main benefit is you can get these metrics for "free" without change
your Spark application.

Thanks and regards,

Ai



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/reuse-the-Spark-SQL-internal-metrics-tp27626.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message