spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bahubali Jain <bahub...@gmail.com>
Subject Dataset : Issue with Save
Date Thu, 16 Mar 2017 17:39:43 GMT
Hi,
While saving a dataset using       *
mydataset.write().csv("outputlocation")  *                 I am running
into an exception



*"Total size of serialized results of 3722 tasks (1024.0 MB) is bigger than
spark.driver.maxResultSize (1024.0 MB)"*
Does it mean that for saving a dataset whole of the dataset contents are
being sent to driver ,similar to collect()  action?

Thanks,
Baahu

Mime
View raw message