It's a bug in Spark old version, In spark 1.6.0, it is fixed.
After the application complete, spark master will load event log to memory, and it is sync because of actor. If the event log is big, spark master will hang a long time, and you can not submit any applications, if your master memory is to small, you master will die!
The solution in spark 1.6 is not very good, the operation is async
, and so you still need to set a big java heap for master.
------------------ 原始邮件 ------------------
发件人: "Shixiong(Ryan) Zhu";<email@example.com>;
发送时间: 2016年3月1日(星期二) 上午8:02
收件人: "Sumona Routh"<firstname.lastname@example.org>;
主题: Re: Spark UI standalone "crashes" after an application finishes
Do you mean you cannot access Master UI after your application completes? Could you check the master log?