spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "lokesh.gidra" <>
Subject Re: "Spilling in-memory..." messages in log even with MEMORY_ONLY
Date Sun, 27 Jul 2014 17:29:14 GMT
I am comparing the total time spent in finishing the job. And What I am
comparing, to be precise, is on a 48-core machine. I am comparing the
performance of local[48] vs. standalone mode with 8 nodes of 6 cores each
(totalling 48 cores) on localhost. In this comparison, the standalone mode
outperforms local[48] substantially. When I did some troublshooting using
oprofile, I found that local[48] was spending much more time in writeObject0
as compared to standalone mode.

I am running the PageRank example provided in the package. 

View this message in context:
Sent from the Apache Spark User List mailing list archive at

View raw message