spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From karan alang <karan.al...@gmail.com>
Subject spark-shell not getting launched - Queue's AM resource limit exceeded.
Date Mon, 07 Aug 2017 03:17:10 GMT
Hello - i've HDP 2.5.x and i'm trying to launch spark-shell ..
ApplicationMaster gets launched, but YARN is not able to assign containers.

*Command ->*

./bin/spark-shell --master yarn-client --driver-memory 512m
--executor-memory 512m

*Error ->*

[Sun Aug 06 19:33:29 +0000 2017] Application is added to the scheduler and
is not yet activated. Queue's AM resource limit exceeded. Details : AM
Partition = <DEFAULT_PARTITION>; AM Resource Request = <memory:2048,
vCores:1>; Queue Resource Limit for AM = <memory:6144, vCores:1>; User AM
Resource Limit of the queue = <memory:6144, vCores:1>; Queue AM Resource
Usage = <memory:6144, vCores:3>;

Any ideas on what parameters to change ?

Pls note -> In YARN, the parameter -
*yarn.scheduler.capacity.maximum-am-resource-percent
= 0.9* , AM should have access sufficient to assign container

Mime
View raw message