spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Re: spark-shell gets stuck in ACCEPTED state forever when ran in YARN client mode.
Date Sun, 08 Jul 2018 14:40:47 GMT
Hi,

It's on local mac book pro machine that has 16GB RAM 512GB disk and 8 vCpu!
I am not running any code since I can't even spawn spark-shell with yarn as
master as described in my previous email. I just want to run simple word
count using yarn as master.

Thanks!

Below is the resource manager log once again if that helps


2018-07-08 07:23:23,343 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue:
Application added - appId: application_1531059242261_0001 user: xxx
leaf-queue of parent: root #applications: 1

2018-07-08 07:23:23,344 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler:
Accepted application application_1531059242261_0001 from user: xxx, in
queue: default

2018-07-08 07:23:23,350 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
application_1531059242261_0001 State change from SUBMITTED to ACCEPTED on
event=APP_ACCEPTED

2018-07-08 07:23:23,370 INFO
org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService:
Registering app attempt : appattempt_1531059242261_0001_000001

2018-07-08 07:23:23,370 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1531059242261_0001_000001 State change from NEW to SUBMITTED

2018-07-08 07:23:23,382 WARN
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
maximum-am-resource-percent is insufficient to start a single application in
queue, it is likely set too low. skipping enforcement to allow at least one
application to start

2018-07-08 07:23:23,382 WARN
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
maximum-am-resource-percent is insufficient to start a single application in
queue for user, it is likely set too low. skipping enforcement to allow at
least one application to start

2018-07-08 07:23:23,382 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
Application application_1531059242261_0001 from user: xxx activated in
queue: default

2018-07-08 07:23:23,382 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue:
Application added - appId: application_1531059242261_0001 user:
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User@
476750cd, leaf-queue: default #user-pending-applications: 0
#user-active-applications: 1 #queue-pending-applications: 0
#queue-active-applications: 1

2018-07-08 07:23:23,382 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler:
Added Application Attempt appattempt_1531059242261_0001_000001 to scheduler
from user xxx in queue default

2018-07-08 07:23:23,386 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1531059242261_0001_000001 State change from SUBMITTED to
SCHEDULED

Mime
View raw message