spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sandudi <phal...@gmail.com>
Subject Spark sortByKey is not lazy evaluated
Date Mon, 12 Feb 2018 12:38:55 GMT
Hey people, sortByKey is not lazy evaluated in spark because of it will use
rang partition but i have one confusion it will load whole data from source
or only sample data. so if i use sortByKey it will get whole data in memory
or only sample so second time when i run real action it will load the whole
data in memory



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message