spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will Benton <>
Subject Profiling Spark tests with YourKit (or something else)
Date Mon, 14 Jul 2014 16:51:11 GMT
Hi all,

I've been evaluating YourKit and would like to profile the heap and CPU usage of certain tests
from the Spark test suite.  In particular, I'm very interested in tracking heap usage by allocation
site.  Unfortunately, I get a lot of crashes running Spark tests with profiling (and thus
allocation-site tracking) enabled in YourKit; just using the sampler works fine, but it appears
that enabling the profiler breaks Utils.getCallSite.

Is there a way to make this combination work?  If not, what are people using to understand
the memory and CPU behavior of Spark and Spark apps?


View raw message