spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mungeol Heo <mungeol....@gmail.com>
Subject How to clean the accumulator and broadcast from the driver manually?
Date Fri, 21 Oct 2016 08:07:29 GMT
Hello,

As I mentioned at the title, I want to know is it possible to clean
the accumulator/broadcast from the driver manually since the driver's
memory keeps increasing.

Someone says that unpersist method removes them both from memory as
well as disk on each executor node. But it stays on the driver node,
so it can be re-broadcast

If it is true, how can I solve the "driver's memory keeps increasing" issue?

Any help will be GREAT!
Thank you.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message