spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abhiguruvayya <sharath.abhis...@gmail.com>
Subject Re: Spark 0.9.1 java.lang.outOfMemoryError: Java Heap Space
Date Fri, 20 Jun 2014 00:58:31 GMT
Once you have generated the final RDD before submitting it to reducer try to
repartition the RDD either using coalesce(partitions) or repartition() into
known partitions. 2. Rule of thumb to create number of data partitions (3 *
num_executors * cores_per_executor). 



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-0-9-1-java-lang-outOfMemoryError-Java-Heap-Space-tp7861p7970.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message