spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Pfeiffer <...@preferred.jp>
Subject Re: spark-submit on YARN is slow
Date Tue, 09 Dec 2014 05:04:23 GMT
Hi,

On Tue, Dec 9, 2014 at 4:39 AM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:
>
> Can you try using the YARN Fair Scheduler and set
> yarn.scheduler.fair.continuous-scheduling-enabled to true?
>

I'm using Cloudera 5.2.0 and my configuration says
  yarn.resourcemanager.scheduler.class =
´┐╝org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler
yarn.scheduler.fair.continuous-scheduling-enabled = true
by default. Changing to a different Scheduler doesn't really change
anything, from ACCEPTED to RUNNING always takes about 10 seconds.

Thanks
Tobias

Mime
View raw message