spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hao Lin <>
Subject Systematically performance diagnose
Date Sun, 29 Dec 2013 06:12:13 GMT
Hi folks,

I am trying to test the performance on a couple of my Spark applications.
For benchmarking purpose, I am wondering if there is a good performance
analysis practice. The best way I can think of is to instrument log prints
and analyze the timestamps in logs on each node.

The major metrics I am interested in are computation ratios (computation
time, data transferring time, basically a timeline of detailed events),
memory usage, disk throughput. Could I have some suggestions on how Spark
is benchmarked.



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message