spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jianneng Li <>
Subject Questions about Spark Shuffle and Heap
Date Fri, 04 Dec 2015 12:01:33 GMT

On the Spark Configuration page (, the documentation
for spark.shuffle.memoryFraction mentions that the fraction is taken from
the Java heap. However, the documentation
for implies that off-heap memory might be
used instead during shuffles. Do these two explanation conflict each other?

In a related question, with Tungsten enabled, when does Spark use off-heap



View this message in context:
Sent from the Apache Spark User List mailing list archive at
View raw message