spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wei Tan <>
Subject long GC pause during file.cache()
Date Sun, 15 Jun 2014 02:24:11 GMT

  I have a single node (192G RAM) stand-alone spark, with memory 
configuration like this in


 In spark-shell I have a program like this:

val file = sc.textFile("/localpath") //file size is 40G

val output = => extract something from line)

output.saveAsTextFile (...)

When I run this program again and again, or keep trying file.unpersist() 
--> file.cache() --> output.saveAsTextFile(), the run time varies a lot, 
from 1 min to 3 min to 50+ min. Whenever the run-time is more than 1 min, 
from the stage monitoring GUI I observe big GC pause (some can be 10+ 
min). Of course when run-time is "normal", say ~1 min, no significant GC 
is observed. The behavior seems somewhat random.

Is there any JVM tuning I should do to prevent this long GC pause from 

I used java-1.6.0-openjdk.x86_64, and my spark-shell process is something 
like this:

root     10994  1.7  0.6 196378000 1361496 pts/51 Sl+ 22:06   0:12 
/usr/lib/jvm/java-1.6.0-openjdk.x86_64/bin/java -cp 

-XX:MaxPermSize=128m -Djava.library.path= -Xms180g -Xmx180g 
org.apache.spark.deploy.SparkSubmit spark-shell --class 

Best regards,

Wei Tan, PhD
Research Staff Member
IBM T. J. Watson Research Center
View raw message