spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeroen Miller <bluedasya...@gmail.com>
Subject Re: Spark on EMR suddenly stalling
Date Tue, 02 Jan 2018 07:50:50 GMT
Hello Gourav,

On 30 Dec 2017, at 20:20, Gourav Sengupta <gourav.sengupta@gmail.com> wrote:
> Please try to use the SPARK UI from the way that AWS EMR recommends, it should be available
from the resource manager. I never ever had any problem working with it. THAT HAS ALWAYS BEEN
MY PRIMARY AND SOLE SOURCE OF DEBUGGING.

For some reason sometimes there is absolutely nothing showing up in the Spark UI or the UI
is not refreshed, e.g. for the current stage is #x while the logs shows stage #y (with y >
x) is currently under way.

It may very well be that the source of this problem lies between the keyboard and the chair,
but if this is the case, I do not know how to solve this.

> Also, I ALWAYS prefer the maximize Resource Allocation setting in EMR to be set to true.


Thanks for the tip -- will try this setting in my next batch of experiments!

JM


---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message