spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ashesh_28 <asheshro...@gmail.com>
Subject Re: Saprk 1.6 Driver Memory Issue
Date Wed, 01 Jun 2016 16:11:25 GMT
Hi Karthik , 

You must set the value before the SparkContext (sc) is created. Also don't
assign too much overhead like 20g for maxResultSize , You can set it to 2G
maximum as per your error message.

Also if you are using Java 1.8 , Please add the below section in your
Yarn-site.xml

 <property>
       <name>yarn.nodemanager.vmem-pmem-ratio</name>
       <value>5</value>
 </property>



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Saprk-1-6-Driver-Memory-Issue-tp27063p27064.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message