spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Muhammad Haris <muhammad.haris.makh...@gmail.com>
Subject Re: Spark Executor Metrics
Date Tue, 16 Aug 2016 15:21:56 GMT
Still waiting for response, any clue/suggestions?


On Tue, Aug 16, 2016 at 4:48 PM, Muhammad Haris <
muhammad.haris.makhtar@gmail.com> wrote:

> Hi,
> I have been trying to collect driver, master, worker and executors metrics
> using Spark 2.0 in standalone mode, here is what my metrics configuration
> file looks like:
>
> *.sink.csv.class=org.apache.spark.metrics.sink.CsvSink
> *.sink.csv.period=1
> *.sink.csv.unit=seconds
> *.sink.csv.directory=/root/metrics/
> executor.source.jvm.class=org.apache.spark.metrics.source.JvmSource
> master.source.jvm.class=org.apache.spark.metrics.source.JvmSource
> worker.source.jvm.class=org.apache.spark.metrics.source.JvmSource
> driver.source.jvm.class=org.apache.spark.metrics.source.JvmSource
>
> Once application is complete, i can only see driver's metrics, have
> checked directories on all the worker nodes as well.
> Could anybody please help me what's i am doing wrong here.
>
>
>
> Regards
>
>
>

Mime
View raw message